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I.  INTRODUCTION  AND  BACKGROUND 

The  quality  of  American  technology  has  enabled  us  to 
successfully  deal  with  difficult  military  conditions  and 
help  minimize  the  precious  loss  of  life.  We  have  given  our 
men  and  women  the  very  best.  And  they  deserve  it. 

President  George  H.  Bush, 

State  of  the  Union  address,  January  1991 


A.  INTRODUCTION 

The  President's  moving  words  of  gratitude  reflect  the 
enormous  success  of  America's  armed  forces  against  Iraq, 
largely  wrought  from  our  overwhelmingly  superior  application 
of  technologies  to  defeat  naked  aggression.  His  stirring 
emotion  had  to  fill  those  responsible  with  quiet  assurance 
that  their  efforts  over  the  past  several  decades  had  been 
vital . 

At  the  same  time,  many  in  Congress  were  and  are  calling 
for  deep  cuts  in  military  spending.  Yet  no  one  in  the 
administration  or  Congress  wants  to  dull  America's 
technological  edge.  How  can  the  U.S.  maintain,  much  less 
improve  its  lead  in  the  race  for  defense  technological 
superiority  under  these  circumstances?  Whatever  the  answer, 
it  will  be  multi-faceted  and  at  least  as  complex  as  the 
bureaucracy  from  which  it  is  born. 

Critical  decisions  in  complex  organizations  are  rarely 
easy,  especially  those  involving  the  expenditure  of 
significant  public  funds  and  with  potential  consequences 
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affecting  the  very  lives  of  the  men  and  women  of  our 
nation's  armed  forces.  The  decisions  about  which 
technologies  are  to  comprise  the  Department  of  the  Army's 
technology  base  investment  strategy  are  central  to  this 
serious  issue. 

Given  these  changes  in  U.S.  budget  priorities  as  well  as 
national  and  global  political  and  economic  realities,  there 
is  certain  to  be  a  call  for  heightened  oversight  and 
accountability  of  the  technology  investment  process.  Thus, 
in  spite  of  defense  management  cries  to  reduce 
micromanagement,  scrutiny  is  sure  to  be  more  strenuous  as 
the  reverse  is  less  politically  attractive  or  rewarding. 

With  increased  scrutiny  of  the  decision  process  will 
come  greater  need  for  clear  logic,  consistency,  and 
credibility  in  technology  investment  decisions  without 
stifling  creativity  from  heightened  risk  aversion.  Analytic 
decision  models  (ADM) ,  designed  to  support  the 
multidimensional  decision  environment  of  defense  technology 
investment,  can  fill  this  need. 

Over  the  past  30  years  there  has  been  something  of  a 
silent  explosion  in  model  development.  Yet  decision  models 
are  not  widely  used  by  the  very  managers  they  are  designed 
to  support  [Ref.  1] .  The  purpose  of  this  study  is  to 
introduce  Army  R&D  managers  and  their  support  staffs  to  the 
virtues  and  limitations  of  some  important  project  selection 
analytic  decision  models  (ADM) .  Given  the  need  and  their 
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effectiveness,  ADM' s  will  have  an  ever  increasing  role  in 
obtaining  logical  solutions  to  problems  of  competing 
priorities,  goals  and  objectives. 

B .  DISCUSSION 

Analytic  decision  models  such  as  the  Decision  Matrix, 
Hurwicz  criterion,  and  Analytical  Hierarchy  Process  are  used 
to  structure  decision  making  under  risk  and  uncertainty  and 
may  offer  some  support  for  the  Army's  research  and 
development  (R&D) /technology  base  investment  strategy 
decision  process.  First  of  all,  formal  decision  models  help 
to  formulate  a  structure  for  all  issues  bearing  on  the 
problem.  They  offer  a  rational  process  for  assigning  values 
to  the  important  factors  on  the  basis  of  current  priorities 
and  policy.  Environmental  factors  such  as  the  likely 
international  political  climate  may  also  weigh  into  the 
problem  and  a  "best"  solution  may  be  deduced  from  the 
relevant,  possible  alternatives. 

Research  into  the  psychology  of  human  judgment  over  the 
past  several  decades  reveals  that,  while  the  environment  in 
which  decisions  are  made  has  become  increasingly  more 
complex,  the  intuitive  judgmental  skills  of  the  decision 
makers  has  not  kept  pace.  Rather  than  clarifying  the 
choices,  ever  increasing  amounts  of  information  often  lead 
to  selectively  "seeing  only  what  we  want  to  see."  Filtering 
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information  in  this  way,  basing  decisions  predominately  on 

past  experience,  can  adversely  skew  results.  [Ref.  2] 

Memory  is  another  weak  point  in  human  judgment.  Since 

many  decisions  are  constructed  from  the  data,  based  upon  a 

reconstruction  of  past  similar  situations,  limited  memory  is 

a  source  of  potential  human  error.  Additionally,  people  do 

not  have  good  success  judging  probabilistic  outcomes 

regarding  risk  and  uncertainty.  People  do  not  have  innate 

intuitive  abilities  to  assess  probable  outcomes  from  random 

input  data.  Formalized  decision  models  can  provide  a 

framework  to  organize  input  data  which  exposes  these  and 

other  sources  of  potential  error,  systematically  minimizing 

risk  to  improve  results.  [Ref.  3] 

Perhaps  the  most  important  reason  for  incorporating  ADMs 

into  the  Army  technology  investment  process  is  money.  The 

Department  of  Defense  (DoD)  annual  budget  will  shrink  each 

year  as  a  result  of  the  President's  agreement  with  the 

Congress  at  the  close  of  1990.  This  trend  is  accelerating  as 

the  apparent  international  military  threat  continues  to 

diminish  and  domestic  economic  needs  mount.  The  Army  will 

have  fewer  dollars  to  spread  around  and  R&D  investment 

decisions  will  undoubtedly  become  more  critical  in  terms  of 

« 

choosing  "the  right"  programs.  The  current  budget  climate  is 
likely  to  intensify  congressional  scrutiny  and  program 
manager  accountability. 
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Steps  to  formalize  the  Army's  technology  investment 
decision  making  could  be  taken  at  any  point  in  the  systems 
research,  development  and  acquisition  (RD&A)  process,  but 
should  first  be  applied  to  the  initial  R&D  investment 
decisions  to  realize  the  greatest  benefit.  Given  that  the 
Army's  Fiscal  Year  (FY)  1992  investment  in  basic  and  applied 
R&D  will  represent  14.4  percent  of  the  Department  of 
Defense's  $40  billion  research  and  development  budget,  even 
small  improvements  in  effectiveness  will  yield  substantial 
expenditure  results.  [Ref.  4] 

A  formalized  decision  process  using  ADMs  can  structure 
the  decision  process  to  reveal  the  logical  steps  used  to 
arrive  at  an  investment  choice.  Such  procedures  do  not 
replace  the  decision  maker.  They  provide  a  record  of  the 
reasoning  used  to  arrive  at  a  decision.  ADMs  can  incorporate 
current  priorities  and  readily  adapt  to  ever  changing 
priorities,  both  planned  and  unplanned,  funding  decisions, 
lessons  learned,  and  shared  experience  from  other  programs 
related  to  the  current  decision.  This  can  ensure  that  no 
single  factor  or  factors  are  unduly  weighted  or  ignored  in 
the  decision  and  that  decision  maker  bias  is  minimized. 

Employing  a  formal  decision  procedure  using  decision 

« 

models  may  add  stability  and  continuity,  thus  reassuring  the 
decision  maker  and  his  superiors.  One  of  the  findings  of  the 
1986  Packard  Commission  and  others  assessing  the  failures  of 
the  defense  acquisition  process  has  been  the  continual 
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turnover  in  defense  programs,  resulting  in  inconsistent 
management  and  program  turmoil.  In  his  1988  book, 

The  Defense  Management  Challenge:  Weapons  Acquisition, 

Ronald  Fox  states  that. 

In  the  late  1980's,  military  rotation  and  individual 
military  career  considerations  caused  program  managers 
to  be  assigned  to  programs  for  an  average  period  of 
only  thirty  months,  and  the  turnover  is  often  higher 
one  or  two  levels  below  the  program  manager.  [Ref.  5, 
pp. 177-178] 

Use  of  ADMs  could  help  to  minimize  the  turmoil  caused  by 
personnel  turnover  by  giving  incoming  managers  awareness  of 
the  logical  progression  of  the  choices  that  brought  the 
program  office  to  its  present  position. 

Once  incorporated  in  the  overall  decision  process,  ADMs 
could  also  lower  the  management  learning  curve  aiding 
continuity  and  further  diminishing  turmoil  and  confusion.  As 
a  potential  source  of  increased  efficiency,  ADMs  might 
reduce  the  perceived  need  for  micromanagement  by  increasing 
management  confidence  at  all  levels  to  which  it  was  applied. 

These  are  only  some  of  the  inefficiencies  that  ADMs  were 
designed  to  address.  If  resolving  these  inefficiencies  is 
truly  a  central  goal  of  the  present  defense  acquisition 
process  reform,  formalizing  the  investment  decision 
procedures  using  analytic  or  economic  decision  models  may  be 
one  small  step  toward  accomplishing  a  major  part  of  that 
goal.  In  any  case,  helping  the  Army's  R&D  investment 
managers  consistently  minimize  the  risk  and  uncertainty  of 
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their  program  choices  may  prove  incentive  enough  for  the 
adoption  of  ADMs . 

C.  SCOPE  OF  THE  THESIS 

The  aim  of  the  study  is  to  develop  a  basic  understanding 
of  what  decision  support  models  are  supposed  to  do  for  the 
decision  maker,  their  usefulness,  and  limitations  for 
research  and  development  project  selection  decisions.  The 
research  study  will  begin  by  describing  the  basic  process  of 
initiating  research  and  development  projects  and  programs  in 
the  buying  commands  of  the  Department  of  the  Army  (DA) . 
Rather  than  examining  how  the  decisions  get  approved  in  the 
budget  submission  and  Program  Objective  Memorandum  process, 
this  work  looks  at  how  the  new  program  initiation  decisions 
are  made  at  the  lowest  level,  where  operational  concerns 
have  some  degree  of  priority  over  budgets  and  bureaucracy. 

The  focus  is  on  the  application  of  ADMs  for  project 
selection  decisions  in  the  area  of  basic  and  applied 
research.  It  is  hoped  that  the  work  will  provide  a  single 
reference  point  for  the  beginning  user  on  the  availability, 
usefulness,  application,  and  limitations  of  proven  decision 
support  models  for  project  selection  problems. 

The  goal  is  to  assess  currently  successful  applied 
economic  and  analytical  decision  models  for  use  by  the 
Army's  RD&A  community.  It  is  designed  to  aid  those  within 
the  Department  of  the  Army  (DA)  charged  with  seeking 
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technology  initiatives  and  promising,  innovative,  high 
payoff  opportunities  to  support  the  transition  from  theory 
to  application. 

Currently  the  Army's  technology  investment  program 
decisions  are  predominantly  made  on  the  basis  of  the  Army's 
Staff  Study  procedure,  whereby  all  viable  alternatives  are 
presented  to  the  decision  maker  with  their  apparent 
advantages  and  disadvantages,  based  upon  stated  or  implied 
criteria,  with  a  recommendation  for  the  "best"  possible 
choice.  The  techniques  introduced  in  this  research  provide 
those  who  must  prepare  such  studies  with  a  model  for 
structuring  each  alternative's  advantages,  disadvantages  and 
criteria  in  order  to  assess  probable  outcomes  upon  which  the 
decision  maker  can  logically  evaluate  the  choices  with  more 
than  intuition  alone. 

D .  METHODOLOGY 

This  study  presents  information  collected  through  a 
search  of  the  current  literature  regarding  DA  policies  for 
program/project  initiation  procedures  to  describe  the 
current  technology  investment  decision  process  in  the 
context  of  the  Department  of  the  Army's  overall  research, 
development,  and  acquisition  cycle.  This  information  is 
supplemented  by  telephone  or  personal  interviews  with  the 
authors  of  the  Army  Technology  Base  Master  Plan:  the  Deputy 
for  Technology  and  Assessment,  Office  of  the  Assistant 
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Secretary  of  the  Army  (R,D&A),  and  follow  up  with 
representatives  of  the  specific  buying  commands  within  the 
Army  Materiel  Command,  such  as,  the  U.S.  Army  Missile 
Command  (MICOM) ,  Tank  and  Automotive  Command  (TACOM) , 
Aviation  Systems  Command  (AVSCOM)  and  others.  [Ref.  6] 

The  literature  search  provided  some  of  the  background 
for  an  analysis  of  the  application  of  analytic  decision 
models  based  on  the  types  of  decisions  being  made  and  the 
factors  typically  bearing  on  relevant  decision  issues. 
Subsequent  analysis  then  details  four  prevalent  analytic 
decision  models  for  multiple  criteria  problems  that  have  a 
significant  record  as  effective  aids  to  decision-making  in 
fields  closely  if  not  directly  related  to  Army  research, 
development  and  acquisition  (RD&A)  program/project  selection 
activities . 

The  data  for  this  analysis  will  be  derived  from  a  review 
of  the  works  on  the  subject  as  well  as  personal  interviews 
with  professors  at  the  Naval  Postgraduate  School  and  other 
experts  in  the  field  of  operations  research.  This  report  is 
not  a  lengthy  scientific  investigation  into  the  worthiness, 
efficiency,  or  absolute  effectiveness  of  each  analytic 
decision  model  uncovered  in  the  course  of  this  research. 
Instead,  it  presents  scholarly  opinion  and  advocacy  of  a 
particular  model's  usefulness,  situation  applicability, 
limitations,  pitfalls,  and/or  any  assumptions  of  those 
experts  who  have  used  and/or  developed  subject  models  based 


9 


on  both  their  written  works  as  well  as  personal  or  telephone 
interviews . 

The  aim  is  to  develop  clear  and  sufficient  evidence  and 
examples  to  allow  managers  of  the  Army's  research  and 
development  investment  strategy  and  their  staffs  to  decide 
with  reasonable  confidence  the  usefulness  of  a  particular 
decision  model  to  their  circumstances.  Each  model  catalogued 
will  be  sufficiently  detailed  to  permit  the  user  to  examine 
and  articulate  the  logic  of  the  model's  structure  and 
application . 
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IX.  THE  ARMY  RESEARCH  AND  DEVELOPMENT  INVESTMENT  STRATEGY 


...with  limited  resources,  an  ever  widening  range  of 
technology  opportunities,  rising  international  tech¬ 
nological  competition,  and  an  increasingly  more  capable 
third  world  threat,  the  Army  must  invest  where  the 
warfighting  requirements  and  potential  improvements  are 
greatest.  [Ref.  6] 


A.  INTRODUCTION 

There  are  certainly  many  reasons  to  consider  improving 
the  efficiency  and  effectiveness  of  Army  R&D  investment 
decisions.  This  chapter  will  explore  the  two  central  to  our 
discussion  thus  far.  First,  with  reductions  in  the  perceived 
threat  to  American  security  abroad  and  the  accompanying 
reduction  in  defense  spending,  some  in  Congress  have 
expressed  serious  concerns  about  assuring  America's 
international  preeminence  in  advanced  defense  technologies 
development  and  assuring  the  maintenance  of  the  defense 
industrial  base.  The  opening  quotation  serves  to  sum  up  the 
Army's  understanding  of  these  concerns. 

Beginning  with  Public  Law  101-189,  November  29,  1989, 
Congress  directed  the  Secretary  of  Defense  (SecDef)  to 
submit  an  annual  plan  for  developing  those  technologies 
considered  most  critical  to  U.S.  defense  qualitative 
superiority.  This  in  turn  led  to  the  Army's  development  of 
its  annual  Technology  Base  Master  Plan  ( TBMP )  to  incorporate 
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the  guidance  from  the  Secretary's  Defense  Critical 


Technologies  Plan.  The  Army  TBMP  provides  the  basis  for  the 
way  in  which  R&D  decisions  are  to  be  made  in  the  Army  in 
light  of  these  congressional  concerns.  [Ref.  7] 

Secondly,  as  described  in  the  opening  background 
discussion,  decision  theory  provides  a  sound  basis  for  using 
decision  models  in  support  of  multiple  criteria  decision 
problems  such  as  those  posed  by  the  Army's  R&D  investment 
process.  These  models  may  provide  a  logical  foundation  for 
structuring  the  analysis  by  incorporating  ADMs  for  specific 
decision  alternative  choices.  Thus,  this  chapter  describes 
the  Army's  research  and  development  investment  decision 
process,  followed  in  Chapter  III  by  an  explanation  of  basic 
decision  making  theory. 

B.  THE  TECHNOLOGY  BASE  INVESTMENT  STRATEGY 

The  decision  process  whereby  the  Department  of  the  Army 
(DA)  determines  which  key  emerging  technologies  will  be 
funded  for  possible  concept  development  is  a  critical  step 
in  shaping  the  material  warfighting  capabilities  of  the 
future  Army.  The  decision  to  fund  or  shelve  an  emerging 
technology  can  have  far  reaching  implications  for  those 
firms  focused  on  defense  applications  as  well  as  society  at 
large  if  the  technology  had  proven  useful  in  civilian 
applications.  Another  concern  is  that  a  critical  link  in  our 
nation's  chain  of  defense  might  be  chopped  from  the  Defense 
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Program  Objective  Memorandum  (POM)  because  of  the  budget 
cutter's  ax. 

The  hard  choices  of  what  flies  and  what  dies  is  sure  to 
leave  many  defense  contractors  gasping  their  final  breath  in 
the  1990s.  Should  DA's  decisions  to  fund  certain  development 
programs  be  based  solely  on  the  needs  of  our  ground  forces 
to  overcome  a  foreign  threat?  Or,  should  the  Army's 
leadership  seek  to  include  the  broader  goals  of  supporting  a 
strong  industrial  base  ready  and  able  to  deliver  the 
American-made  war  goods  needed  for  the  next  theater  of 
conflict  as  well  as  the  broad  socioeconomic  legislation  all 
government  procurement  is  pledged  to  support.  The  issue  of 
supporting  the  technology  base  has  already  been  partially 
addressed  in  Congress  and  the  Army  Acquisition  Executive, 
Stephen  K.  Conver  has  acknowledged  that  goal.  [Ref.  8] 

One  thing  is  certain:  as  long  as  the  Army's  Procurement 
budget  continues  to  shrink  from  $14.2  billion  in  1991,  down 
to  $11.6  billion  in  1992,  and  projected  under  $9.5  billion 
for  1993,  the  choices  will  become  more  difficult  and  the 
consequences  of  an  inefficient  choice  more  severe.  For,  in 
strictly  economic  terms,  less  Defense  Procurement  spending 
means  less  diversification  over  fewer  systems  and  less 
private  research  and  development  spending  by  contractors 
strapped  by  reductions  in  Defense  Procurement  spending. 

[Ref.  9] 
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The  shrinking  Army  budget  for  fiscal  years  '92  and  '93 
will  have  a  serious  impact  on  Army  R&D  investment.  While  the 
Army's  investment  in  basic  and  exploratory  research  is 
growing  in  response  to  Congressional  concerns,  compared  to 
the  rest  of  DoD,  the  Army's  share  is  declining.  For  example, 
while  the  Army's  share  of  the  total  defense  budget  for 
Fiscal  Year  '93  is  projected  to  be  just  under  25  percent, 
the  Army  share  of  the  total  DoD  RDT&E  program  is  projected 
to  decline  from  15.5  to  14.3  percent.  A  credible  technology 
research  investment  strategy  will  be  essential  to  assure  a 
viable  technology  base  for  future  Army  systems  and  could  be 
a  catalyst  to  reverse  this  trend.  These  factors  increase  the 
Army's  technology  risk  in  several  ways.  [Ref.  4] 

One  risk  is  the  shrinking  industrial  and  technology 
base.  With  declining  defense  spending  spread  over  fewer 
program  new-starts,  defense  contractor  funded  basic  research 
is  expected  to  fall.  Fewer  choices  between  contractors  and 
programs  will  mean  that  some  key  prime  contractors  may  leave 
the  field  and  with  them  may  go  critical  expertise.  This 
concern  is  perhaps  even  more  acute  for  the  lower  tiered  sub¬ 
contractors  who  support  the  primes  with  precision  science 
and  engineering  skills.  [Ref.  10] 

We  learned  in  the  70s  and  80s  that  limited  deployment 
times  meant  "come-as-you-are-war"  with  no  time  to  gear  up 
industry  to  overcome  our  material  shortcomings.  The 
alternative  may  be  to  rely  on  the  capabilities  of  allies  for 
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support  in  those  areas  in  which  we  fall  behind.  Obviously 
this  is  in  part  what  Congress  hopes  to  avoid  by  supporting 
the  Defense  Critical  Technologies  Plan. 

In  sum,  there  may  not  be  time  to  recover  in  a  crisis. 
There  may  not  be  the  expertise  to  recover.  We  may  not  have 
the  resources  to  commit  or  divert  to  such  a  recovery  effort 
in  time  to  meet  the  need.  Reliance  on  allies  leaves  us 
vulnerable  in  other  ways.  There  is,  as  there  has  always 
been,  the  potential  that  some  important  breakthroughs  will 
be  overlooked  entirely  in  the  decision  process,  but  this 
risk  increases  as  fewer  dollars  are  spent  on  fewer  programs. 

Before  suggesting  some  ways  to  minimize  these  potential 
risks  in  the  decision  process,  it  is  appropriate  to  review 
the  way  in  which  the  Army  selects  research  technologies  in 
which  to  invest  will  be  presented.  Although  the  entire 
process  is  fairly  complicated  with  many  decision  makers  at 
various  levels,  the  following  is  a  basic  summary. 

C.  THE  ARMY  RESEARCH  AMD  DEVELOPMENT  INVESTMENT  PROCESS 

The  Army  has  divided  its  research  program  into  four 
commands:  the  Army  Materiel  Command  (AMC) ,  the  Deputy  Chief 
of  Staff  for  Personnel  (DCSPER) ,  the  Surgeon  General,  and 
the  U.S.  Army  Corps  of  Engineers.  Each  has  responsibility 
for  developing  technologies  critical  to  their  area  of 
expertise  and  each  has  a  structure  for  managing  various 
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aspects  of  the  technology  base  to  meet  the  needs  of  their 
user  community.  [Ref.  11] 

For  example,  AMC  includes  Laboratory  Systems  Command 
(LABCOM)  with  leadership  over  seven  Army  Research, 
Development  and  Engineering  Centers  (RDEC)  and  the  Army 
Research  Office  (ARO) .  The  Army  laboratory  structure 
includes  42  laboratories  and  RDECs ,  complicating 
communication  and  technology  transfer.  A  LABCOM  plan  for 
restructuring  the  Army  laboratory  system,  called  LAB-21,  has 
been  included  in  the  1990  Base  Closure  and  Realignment  Act. 
It  is  a  six  year  program  designed  to  streamline  and  increase 
the  productivity  and  efficiency  of  the  Army's  research  and 
development  organizations.  [Ref.  12] 

The  ARO  on  the  other  hand,  interacts  with  the  nation's 
research  universities,  funding  basic  research  in  university 
laboratories  throughout  the  country.  Any  of  these  actors  can 
play  a  part  in  the  process  that  determines  which 
technologies  should  be  funded  for  further  development  in 
this  highly  decentralized  management  structure. 

The  formal  decision  process  also  includes  the  "user 
community"  represented  by  the  Army's  Training  and  Doctrine 
Command  (TRADOC)  which  has  leadership  over  each  of  the  Army 
combat  training  branches,  from  Artillery,  Aviation,  Armor, 
to  the  Ordnance  Corps,  the  Signal  Corps,  and  Medical  Corps. 
Each  of  TRADOC 's  combat  and  combat  support  training  schools 
publishes  their  annual  Mission  Area  Analysis  (MAA)  in  which 
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they  identify  their  greatest  combat  survivability  and 
offensive  capabilities  needs  based  on  TRADOC's  warfighting 
doctrine  for  the  Army  and  the  threat  capabilities 
assessment . 

TRADOC  then  consolidates  and  prioritizes  these  needs  and 
hands  them  off  to  the  Secretary  of  the  Army.  The  Army's 
Joint  Requirements  Oversight  Council,  chaired  by  Army 
Acquisition  Executive,  Steven  Conver,  Assistant  Secretary  of 
the  Army  (Research,  Development,  and  Acquisition)  marries 
requirements  with  capabilities  and  funding.  The  result  is 
the  Army's  Technology  Base  Master  Plan  (TBMP)  which  includes 
their  Technology  Base  Investment  Strategy.  [Ref.  13] 

According  to  the  1990  Army  TBMP,  thirteen  areas  are 
considered  critical  to  the  Army's  future  warfighting 
capabilities.  They  are  called  the  Army's  key  emerging 
technologies.  The  list  includes  directed  energy  weapons, 
biotechnology,  artificial  intelligence,  neuroscience, 
robotics,  and  others.  Each  of  the  thirteen  "were  reviewed  by 
Army  technical  managers,  scientists,  and  engineers.  Then 
Army  leadership  approved  the  final  list."  [Ref.  5] 

D .  THE  ARMY  TECHNOLOGY  BASE  MASTER  PLAN 

The  Army  TBMP  represents  the  Army  leadership's  top  down 
guidance,  objectives,  and  goals  and  includes  the  Technology 
Base  Investment  Strategy.  It  is  intended  to 
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...create  an  atmosphere  that  fosters  technology 
initiatives  and  the  pursuit  of  promising  innovative 
opportunities  while  providing  sufficient  flexibility  and 
funding  to  laboratory  and  RDEC  Directors  to  seize  local, 
high  payoff  opportunities.  [Ref.  6,  p.  1-10] 

The  process  of  review  and  approval  of  the  future 
technology  investment  needs,  from  the  MAAs  to  the  TBMP,  is 
based  on  the  experience,  judgment,  and  intuition  of  the 
decision  makers  involved.  It  is  an  annual  process,  beginning 
with  the  RDEC  Directors  and  their  supporting  staffs,  up  to 
the  Assistant  Secretary  of  the  Army  for  Research, 
Development,  and  Acquisition  (RD&A) .  How  the  exact  process 
actually  works  is  not  clearly  spelled  out,  but  one  source 
summarizes  it  this  way: 

In  the  case  of  advanced  technology  transition  demonstra¬ 
tions,  the  Army  employs  a  Senior  Advisory  Group  (SAG)  co¬ 
chaired  by  the  Deputy  Assistant  Secretary  for  Research  and 
Technology,  DASA  (RDA) ,  and  the  Assistant  Deputy  Chief  of 
Staff  for  Operations  and  Plans,  Force  Development,  HQDA. 
Submissions  are  made  to  a  working  group  that  is  co-chaired 
by  the  Army  Materiel  Command  Deputy  Chief  of  Staff  for 
Technology  Planning  and  Management,  and  the  Assistant  Deputy 
Chief  of  Staff  for  Combat  Developments,  TRADOC.  The  working 
group  reviews  candidate  proposals  and  provides  recommen¬ 
dations  to  the  SAG  for  new  start  approval.  This  procedure 
began  in  April  1990  in  response  to  the  Office  of  the 
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Secretary  of  Defense  (OSD)  Critical  Technologies  Plan  (CTP) 
guidance.  [Ref.  14] 

In  addition  to  its  internal  process  of  approval,  all 
Army  programs  have  to  be  funded  in  the  annual  Program 
Objectives  Memorandum  (POM)  approved  by  the  Secretary  of 
Defence  and  the  Joint  Chiefs  of  Staff.  This  step,  POM 
approval,  is  the  juncture  where  a  perceived  need  becomes  a 
budgeted  requirement.  It  is  possible  that  a  critical  Army 
need  (in  the  eyes  of  the  users)  could  go  unbudgeted  in  the 
Army  POM,  conceivably  leaving  our  forces  vulnerable  in  a 
critical  combat  area. 

In  addition  to  the  POM,  OSD  publishes  its  annual  Criti¬ 
cal  Technologies  Plan  for  Congress.  The  Services  are 
expected  to  follow  it  in  designing  their  TBMP's.  The  CTP 
represents  the  Defense  Department's  judgment  of  the  20  most 
important  weapons-related  technologies  of  the  200  or  so 
technologies  recommended  by  the  Services.  Multiple  criteria 
are  used  to  determine  which  technologies  make  the  CTP. 

Finally,  priorities  are  assigned  based  on  the  judgments 
of  a  working  group  with  inputs  from  a  number  of  defense 
industry  contractors.  The  final  decision  is  made  by  a  senior 
committee  representing  individuals  in  the  DoD  and  the 
Department  of  Energy  with  management  responsibility  over  the 
National  Science  and  Technology  program.  [Ref.  7] 

This  is  a  decision  process  steeped  in  the  give  and  take 
of  group  dynamics,  political  coalition  building,  consensus 
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and  compromise  among  key  stakeholders,  their  advocates  and 
adversaries.  There  are  many  actors  and  plenty  of  room  for 
inefficiency,  if  not  outright  error.  The  next  chapter  will 
discuss  additional  sources  of  potential  error  in  decision 
making  that  can  introduce  inefficiencies  of  their  own. 
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III.  BASIC  DECISION  THEORY  CONCEPTS 


A.  INTRODUCTION 

Our  goal  thus  far  in  reviewing  the  Department  of  the 
Army  technology  investment  strategy  has  been  to  demonstrate 
that  the  process  is  not  immune  to  the  risks  of  human 
decision  making  and  their  resulting  inefficiencies.  This 
chapter  will  be  a  presentation  of  basic  decision  theory 
fundamental  to  an  understanding  of  the  importance  of 
decision  model  application.  The  discussion  will  conclude 
with  an  evaluation  of  the  applicability  of  incorporating 
analytic  decision  models  into  the  DA  process  for  potential 
improvements  in  efficiency  and  effectiveness. 

It  should  be  noted  that  although  there  are  many  human 
dynamics  theories  and  models  (such  as  the  rational  policy 
model,  the  organizational  process  model,  the  bureaucratic 
politics  model,  among  others)  affecting  the  process  thus 
described,  their  assessment  is  outside  the  scope  of  this 
current  work.  While  these  human  dynamics  models  may  be 
essential  to  the  implementation  of  organizational  decisions, 
they  have  little  to  do  with  analyzing  and  achieving  economic 
efficiency . 
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B.  THE  ROOTS  OF  ANALYTIC  DECISION  THEORY 

The  study  of  decision  making  under  uncertainty  has  a 
long  past.  Beginning  in  the  1600 's  with  the  founders  of 
modern  probability  theory,  Fermat  and  Pascal,  it  was  shown 
that  outcomes  over  time  could  be  quantified  by  the 
likelihood  of  their  occurrence.  This  probability  of 
occurrence  times  its  known  payoff  can  be  summed  to  support  a 
choice  on  the  basis  of  the  outcome's  expected  value,  which 
is  the  average  of  all  the  potential  payoffs.  This  idea  led 
to  the  belief  that  if  we  can  quantify  the  payoffs  and  their 
likelihood  of  occurrence,  rational  decision  makers  could  use 
this  information  in  judgments  involving  uncertainty. 

[Ref .15] 

Decision  making,  however,  is  much  more  complex.  The 
complexity  of  rational  decision  making  under  uncertainty  is 
supported  by  a  vast  field  of  research  on  the  psychological 
profile  of  decision  makers  and  spans  the  topics  of 
heuristics  and  biases,  prediction  theory,  inductive 
reasoning,  causal  schemata,  conservatism,  and  risk 
perceptions,  to  name  just  some  of  the  broad  categories. 

Probably  the  most  important  work  regarding  the  way 
managers  make  decisions  is  being  done  by  Dr.  Herbert  A. 
Simon,  professor  of  Computer  Science  and  Psychology  at 
Carnegie-Mellon  University.  Professor  Simon's  work  on  how  we 
make  decisions  in  organizations  spans  more  than  40  years  and 
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earned  him  the  Nobel  Prize  in  1978  for  his  concepts  of 
"objective,  subjective,  and  bounded  rationality.”  [Ref.  16] 
Basically,  Professor  Simon  logically  developed  the 
notions  of  rational  decision  making  and  pointed  out  many 
important  shortcomings  in  human  cognitive  abilities  that 
limit  our  ability  to  make  rational  decisions.  For  example, 
according  to  Professor  Simon,  people; 

•  tend  to  limit  the  scope  of  the  decision  problem,  the 
factors  impacting  the  choices,  the  alternative  pros  and 
cons,  and  objective  criteria  in  some  way  to  'bound'  the 
issue  within  their  capacity  and  experience,  discounting 
data  that  doesn't  conform  to  closely  held  beliefs. 

•  do  not  seek  to  optimize  in  their  decision-making  process 
but,  rather  seek  to  select  solutions  that  are  good 
enough . 

•  attempt  to  construct  means-ends  (or  cause-effect)  chains 
to  solve  complex  problems  in  a  series  of  steps  to 
achieve  the  ultimate  ends  in  a  hierarchical  fashion. 
However,  the  conscious  integration  of  weighing  values 
against  alternatives  often  becomes  an  increasingly 
complex  chain  and  is  seldom  attained  in  actual  behavior. 

•  judge  alternative  strategies  based  on  their  experience 
and  prior  opinions  about  the  consequences  that  follow  on 
their  choice  of  action.  Obviously,  no  one  can  directly 
know  the  consequences  of  all  behaviors.  So,  the  quality 
of  any  decision  depends  on  how  close  their  experience 
corresponds  to  the  choice  problem. 

•  often  weigh  information  received  initially  and  lastly 
more  heavily  than  information  received  in  the  middle  of 
a  discussion  on  alternatives  about  which  they  must 
formulate  a  decision. 

•  tend  to  classify  problems* as  programmed  and  non- 
programmed.  In  organizations  the  former  tends  to  drive 
out  the  latter.  Non-programmed  decisions  are  generally 
solved  by  reducing  them  to  a  series  of  programmed 
decisions.  [Ref.  17] 
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One  distinguished  source  on  Simon's  work  made  the 
following  observation: 

...  it  is  essentially  impossible  to  attain  objective 
rationality.  To  do  so  the  decision-maker  must  know  all 
alternatives,  construct  all  appropriate  means-ends 
chains,  assess  expectations  for  all  uncertain 
consequences,  etc.  It  is  here  that  Simon's  notion  of 
bounded  rationality  takes  form.  The  decision-maker  can 
know  these  important  aspects  of  a  decision  only  within 
the  limits  imposed  by  experience,  knowledge,  time  and 
effort  available  to  search  and  study  the  decision 
environment.  [Ref  16,  p.  39] 

So  where,  one  might  ask,  does  that  leave  the  rest  of  us 

mere  mortals?  The  answer  lies  in  Simon's  notion  of 

subjective  rationality.  For  a  decision  to  be  subjectively 

rational,  it  must: 

•  be  the  best  choice  dictated  by  the  current  state  of 
knowledge  using  all  available  data 

•  be  congruent  with  the  values,  goals,  and  objectives  of 
the  organization  with  regard  to  the  decision  criteria 

•  conform  to  the  organization's  subject  experts' 
collective  judgment  regarding  alternative  consequence 
outcomes.  [Ref.  16,  p.  40] 


None  of  this  may  be  particularly  earth  shattering  to 
today's  manager,  but  Professor  Simon's  1977  book.  The  New 
Science  of  Management  Decision  [Ref.  17],  brought  many  of 
these  interdisciplinary  concepts  of  decision  theory  together 
for  the  first  time.  It  not  only  identified  the  basic  human 
decision  process  and  its  foibles,  but  also  provided  some 
exceptional  direction  for  resolving  non -programmed  decisions 
in  organizations.  It's  an  exceptionally  positive  book,  in 
that  regard.  It  should  be  required  reading  for  any  manager 
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preparing  to  confront  multiple  criteria  decision  problems. 
Its  168  pages  offer  extensive  management  insight  that  might 
otherwise  take  years  of  experience  to  attain. 

1  C.  RISK  CONCEPTS  IN  DECISION  MAKING 

No  discussion  of  the  basics  of  decision  theory  is 

4 

complete  without  a  comment  regarding  risk.  After  all,  no 
choice  among  competing  alternatives  would  be  difficult  if 
everyone  knew  with  certainty  the  outcomes  of  each 
alternative.  The  decision  maker  (DM)  would  choose  the 
alternative  whose  outcome  best  achieved  the  organization's 
goals,  assuming  the  DM  were  rational.  The  above  discussion 
of  objective  and  subjective  rationality  sheds  light  on  the 
ways  in  which  DMs  deal  with  uncertainty  in  complex  problems 
given  human  cognitive  limits.  Risk  management  is  another 
important  aspect  of  the  R&D  project  selection  decisions. 

The  concept  of  risk  is  closely  related  to  the  notion  of 
uncertainty.  In  terms  of  decision  making,  risk  is  the  degree 
of  certainty  that  an  undesirable  outcome  will  occur  as  the 
result  of  an  action  or  event.  To  clarify  this  from  the 
common  notion  of  risk,  think  of  it  this  way.  We  could  say, 
for  example  that  the  risk  of  cancer  among  white  males  in  the 
U.S.  is  25  percent,  meaning  that  roughly  25  percent  of  all 
•  white  males  will  contract  cancer  in  their  lifetime.  The 
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degree  of  uncertainty  determines  the  level  of  risk.  If  we 
know  an  outcome  is  certain  to  result  from  an  action,  there 
is  no  risk  associated  with  the  decision. 

The  confusion  comes  when  we  think  of  the  popular  notion 
of  taking  action  which  reduces  the  uncertainty  of  a  negative 
outcome  as  placing  us  at  a  "higher  risk."  To  continue  the 
cancer  analogy,  those  who  smoke  all  their  lives  actually 
lower  the  uncertainty  that  they  will  become  cancer  victims, 
yet  we  say  they  are  at  higher  risk.  In  fact,  we  can  predict 
with  reasonable  certainty  that  they  will  indeed  contract 
cancer.  This  popular  notion  of  risk  actually  shifts  from  an 
event  uncertainty  to  a  timing  and  security  uncertainty 
involved  with  perception  and  fear.  [Ref.  18] 

In  project  selection,  the  risk  of  an  alternative  is 
weighed  against  a  subjective  appraisal  that  its  payoff  will 
exceed  the  resources  needed  to  achieve  the  anticipated 
result .  The  appraisal  must  include  the  concept  of 
opportunity  cost  (the  payoffs  foregone  from  the  most 
promising  alternative  opportunities)  to  be  valid.  The 
greater  the  potential  payoff  of  an  alternative,  the  greater 
the  acceptable  risk,  within  the  resource  limits  imposed. 
[Ref.  19] 

Even  the  novice  DM  is  basically  familiar  with  this 
concept  of  risk  in  choosing  among  alternatives. 

Nevertheless,  the  impact  of  risks  and  their  effects  on 
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choice  cannot  be  dismissed.  Particularly  in  austere  budget 
climates  when  risk  aversion  is  likely  to  increase. 

Risk  averse  behavior  is  a  judgment  bias  that  can 
severely  limit  innovation  in  the  technology  investment 
process  and  must  be  guarded  against  as  any  bias  should  be  in 
rational  decision  making.  Some  of  the  decision  models  that 
will  be  examined  in  the  upcoming  chapters  are  specifically 
designed  to  highlight  risk  levels  to  help  the  DM  avoid  the 
inefficiency  resulting  from  risk  averse  behaviors. 

D.  THE  DECISION  ENVIRONMENT 

One  final  concept  regarding  complex  decisions  is  the 
notion  of  the  decision  environment  encompassing  the 
organization  making  the  decision.  Fundamental  changes  in  an 
organization's  internal  and/or  external  environment  can 
dramatically  alter  the  organization's  mandates,  goals  and 
obj ectives-the  basis  on  which  important  decisions  rest.  In 
the  case  of  the  Army's  R&D  technology  investment  decisions, 
environment  refers  to  the  national  and  international 
strategic,  economic,  and  geo-political  setting.  As  one 
research  scientist  pointed  out  in  this  regard,  "The  decision 
space  is  Markovian:  Given  today,  the  past  and  future  are 
independent  [Ref.  20].“ 

The  point  is,  as  the  Russian  statistician,  Markov, 
theorizea,  the  future  state  of  nature  depends  only  on  the 
present  environment  and  cannot  be  accurately  ascertained 
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from  the  past  because  the  future  is  independent  of  past 
events.  They  are  connected  only  by  a  sequence  of  events  in 
time.  Because  the  outcome  payoff  of  a  course  of  action  is 
directly  linked  to  the  environment  in  which  it  results,  the 

9 

DM's  payoff  assessment  must  incorporate  his  belief  of  the 

future  state  of  nature  at  the  time  that  the  program  achieves  * 

maturity.  [Ref. 20] 

This  notion  is  especially  pertinent  to  a  development 
cycle  conservatively  estimated  to  take  20  years  from  concept 
to  fielding.  It  is  this  notion  of  Markovian  independence 
that  has  given  operations  research  its  broad  footing  and 
support  for  the  development  of  decision  models.  No  other 
discipline  copes  with  this  concept  in  a  decision  context. 

E.  DECISION  THEORY  SUMMARY 

Risk,  payoff,  uncertainty,  rationality,  opportunity 
cost,  judgment,  all  relate  to  statistical  probabilities  in 
choosing  among  outcomes,  which  can  leave  all  but  the 
operations  research  analyst  more  than  a  little  nauseous.  The 
point  in  providing  decision  theory  is  to  help  Army  R&D 
managers  make  better,  informed  choices  supported  by 
technical  analysis  in  their  organizations,  not  to  be  better 
statisticians . 

Even  if  no  decision  models  are  used  to  aid  the  decision  ♦ 

process,  keeping  human  cognitive  errors  in  mind  can  improve 
decision  quality.  Models  were  never  developed  to  replace 
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human  judgment  but  to  enhance  its  capacity.  The  following 
judgment  biases  and  pitfalls  are  discussed  in  chapter  I: 


•  filtering  bias 

•  experience  bias 

•  to  give  priority  to  information  by  order  received 

•  memory  capacity  limitation 

•  conformity  to  prior  beliefs 

•  causal  chains  of  likely  outcomes  limitation 

•  risk  preference  bias 

•  independence  of  the  future  from  the  present 
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IV.  INTRODUCTION  TO  ANALYTIC  DECISION  MODELS 


. . .  when  asked  to  advise  companies  on  the  acquisition 
of  a  computer,  my  advice  was,  before  any  commitment, 
make  a  careful  determination  of  its  need  and  how  they 
would  use  it.  I  soon  realized  that  this  was  poor  advice- 
a  company  only  acquired  the  ability  to  make  sound  deci¬ 
sions  about  computers  by  hands-on  experience  with  them. 

The  first  investment  in  a  computer  was  not  to  be  judged 
by  its  cost  saving  potential-it  might  have  none-but  by 
its  contribution  to  intelligence  capabilities  and  subse¬ 
quent  decisions. 

Dr.  Herbert  A.  Simon,  1977 

The  New  Science  of  Management  Decision,  p.  43 
Dr.  Simon's  comment  regarding  his  early  advice  to 
companies'  attainment  of  increased  "intelligence 
capabilities"  can  apply  equally  well  today  to  multi-criteria 
decision  support  models.  These  two  powerful  management 
tools,  decision  models  and  computers,  have  a  nearly  parallel 
development  history.  Decision  support  models  of  any 
complexity  are  almost  completely  reliant  upon  computers  for 
their  often  lengthy  computations.  This  can  at  once  serve  to 
enhance  a  model's  considerable  utility  and  further  shroud 
its  seemingly  mystical  results. 

Multicriteria  decision  models  (MCDM)  can  confound  the 
manager  confronted  with  dealing  with  the  models'  results  and 
the  analysts'  technical  interpretation.  Although  they  offer 
the  manager  improved  leverage  in  enhanced  decision  making 
capabilities,  it  has  been  widely  documented  that  management 
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places  minimal  emphasis  on  their  use  in  actual  decision 

making  [Ref.  1  and  21].  Yet,  as  one  important  source  put  it: 

Whether  a  modeler's  recommendations  are  accepted  or 
rejected,  they  must  be  questioned  and  understood.  If 
they  are  rejected,  equally  quantitative  arguments  may  be 
required  to  contradict  the  analysis....  The  planner  who 
uses  computers  and  formulas  must  be  reckoned  with.  [Ref. 
22,  p.l] 

A.  BACKGROUND  ON  GENERAL  DECISION  SUPPORT  MODELS 

Although  many  decision  support  models  have  been 
developed  since  management  science  got  its  start  after  World 
War  II,  many  are  not  well  suited  for  R&D  project  selection 
problems  with  multiple  objectives.  For  example,  Professor 
William  Souder  of  the  University  of  Alabama,  Huntsville,  in 
his  1984  book.  Project  Selection  and  Economic  Appraisal 
[Ref.  23],  summarizes  nearly  the  entire  field  in  150  pages. 
However,  the  models  he  presents  are  those  which  primarily 
focus  on  one  obiective-the  project's  economic  benefit  or 
payoff . 

Economic  decision  models  such  as  cost /benefit  analysis, 
break-even  point,  make  or  buy,  return  on  investment,  payback 
period,  and  internal  rate  of  return  are  all  excellent  and 
may  be  absolutely  essential  to  business  decisions  where 
profits  determine  performance  (if  not  out  right  survival) . 
Others,  like  the  Simplex  method  and  Dynamic  Programming,  use 
a  sequential  approach  to  optimize  resource  allocation.  By 
allocating  the  scarce  resource  in  question  (personnel. 
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space,  machine  time,  funding,  etc.)  to  the  candidate 
projects,  as  many  of  the  candidate  projects  (or  jobs)  are 
completed  as  the  resource  allows. 

Both  single  objective  and  sequential  decision  support 
models  require  that  alternative  action  outcomes  be  known 
with  mathematical  certainty  and  measurable  in  terms  of  the 
subject  resource  (usually  in  dollars) .  In  the  public  sector, 
this  is  frequently  not  the  case.  Many  other  factors  weigh 
into  the  problem  in  a  free  and  democratic  society  from  many 
quarters  and  competing  stakeholders.  From  benefits  to  the 
local  community  to  the  nation's  technology  base,  in  terms  of 
jobs  and  welfare,  to  effects  on  the  environment,  to  issues 
of  keeping  annual  program  budgets  fully  committed  -  many 
issues  must  be  simultaneously  addressed  for  a  decision  model 
to  be  completely  beneficial  to  the  user.  (See,  References 
17,  22,  23  and  24,  if  need  be,  for  an  excellent  development 
of  the  economic  and  analytic  decision  models  mentioned 
here)  . 

Obviously,  if  the  choices  between  alternatives  were 
trivial  there  would  be  no  need  for  detailed  modeling  of  the 
decision  problem.  If  the  DM  is  reasonably  certain  of  the 
outcome  of  each  alternative  project  available  and  suitable 
to  comprise  his  R&D  program  in  a  future  fiscal  period,  the 
individual's  intuition  and  judgment  alone  are  sufficient. 
However,  most  R&D  project  selection  decisions  are  complex 
for  several  reasons. 
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They  often  involve  a  host  of  interrelated  elements  with 
varying  degrees  of  impact  on  the  decision.  Furthermore,  the 
experts  upon  whose  judgments  the  DM  may  have  to  rely,  will 
often  be  stakeholders  in  the  decision's  outcome,  with  their 
own  varied  priorities  weighing  in  their  decision  criteria 
[Ref .  26] . 

Expert  input  is  valued,  but  may  need  to  be  standardized 
or  quantified  to  weigh  considerations  on  a  level  basis,  to 
avoid  lending  undue  credence  to  any  particular  point  of  view 
[Ref.  26].  Essentially,  if  the  trusted  experts  disagree  on 
the  criteria  and  approach  for  weighing  the  value  of 
alternatives,  and  the  outcome  is  critical,  the  decision  is 
non-trivial  and  the  need  to  quantify  qualitative 
considerations  and  factors  to  structure  the  decision  problem 
should  be  apparent. 

B.  THE  MULTI -OBJECTIVE  ENVIRONMENT  OF  R&D  PROJECT  SELECTION 

Consider  for  a  moment  the  Army's  Research,  Development 
and  Engineering  Center  (RDEC)  directors  who  are  expected  to 
balance  the  many  objectives  listed  in  Table  4.1  [Ref.  6] 
from  Congress,  Office  of  the  Secretary  of  Defense  (OSD),  on 
down  to  Army  Materiel  Command  priorities.  In  addition,  there 
are  a  number  of  technology  investment  objectives  that 
conflict  or  complement  these  stated  goals.  Some  current 
examples  include  minimizing  development  time,  life-cycle 


33 


cost,  damage  to  the  environment,  while  at  the  same  time 
maximizing  dual  use  technologies,  performance  capability. 


TABLE  4 . 1  THE  ARMY  TECHNOLOGY  BASE 
STRATEGIES;  Source:  1990  Army 
Technology  Base  Master  Plan  [Ref.  6,  pp. 
1-9  -  1-10] 


♦Enhance  warfighting 

♦Balance  R&D  between  near,  mid,  and 
far-term  needs 

♦Balance  between  next  generation, 
emerging  technologies,  systemic 
issues,  and  support  capabilities 

♦Seize  technology  initiatives 

♦Leverage  R&D  outside  the  Army 

♦Reduce  concept  risk  and  development 
times 

♦Foster  innovation  and  seek  high 
payoff  opportunities 


and  protecting  the  industrial  and  technology  bases,  all 
within  a  projected  budget  limitation  essentially  beyond  his 
or  her  immediate  control. 

This  is  the  decision  environment  multicriteria  decision 
models  (MCDM)  were  developed  to  handle  and  support.  It  is  an 
environment  in  which  the  decision  maker  (DM)  must  achieve 
multiple  goals  with  a  given  budget  over  a  broad  range  of 
continually  changing  R&D  project  alternative  opportunities, 
each  with  its  own  expected  returns  for  achieving  at  least 
some  of  the  required  objectives. 
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How  can  project  alternatives  be  compared  to  determine 
which  ones  best  satisfy  program  decision  criteria?  Where  a 
considerable  budget  is  involved  with  frequently  changing 
goals,  objectives,  criteria  and  priorities,  the  decision 
process  will  hardly  be  routine.  As  mentioned  previously, 
with  increased  scrutiny  and  oversight,  expert  opinion  alone 
may  no  longer  suffice. 

The  basic  answer  to  our  "How  to  compare"  question  lies 
in  quantifying  subjective  judgments  from  experts  in  a 
systematic,  formal  analysis  of  alternatives  and  objectives. 
Some  might  conclude  that  complex  issues  in  the  public  sector 
should  be  left  to  subjective  evaluations,  but  for  many  of 
the  reasons  stated  in  the  second  chapter,  it  is  quite  often 
inefficient  to  do  so. 

For  example,  although  we  may  not  be  able  to  directly 
quantify  the  value  of  several  aircraft  designs  of  near  equal 
cost-  some  offering  improved  performance,  others  increased 
efficiency,  and  still  others  technological  advantages-  we 
could  ask  experts  to  judge  which  design (s)  is  preferred  over 
the  other  and  use  the  preferences  to  develop  a  rank 
ordering.  Quantifying  the  rankings  and  comparing  them 
against  priorities  and  constraints  can  lead  to  a  logical 
choice  among  alternatives.  [Ref.  27] 

In  conclusion,  several  important  multi-criteria  decision 
models  (MCDM)  have  been  developed  to  support  the  kinds  of 
decisions  facing  the  Army  in  its  R&D  project  selection 
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process.  These  are  non-trivial  decisions  involving  extensive 
cost,  payoff  data,  and  opinion  as  well  as  many  competing 
goals,  objectives,  criteria  and  priorities.  By  organizing 
the  input  data,  quantifying  expert  judgments,  and  forcing 
choices  among  competing  criteria,  MCDM  can  help  the  decision 
maker  (DM)  overcome  many  of  the  inefficiencies  inherent  to 
human  decision  making-  namely,  biases  and  error-  discussed 
in  chapter  III. 

C.  PURPOSE  AND  STRUCTURE  OF  THE  UPCOMING  FOUR  CHAPTERS 

The  next  four  chapters  are  designed  for  the  management 
DM  who  may  not  be  aware  of  the  potential  of  MCDM 
applications  for  supporting  R&D  project  selection  decisions 
in  the  public  sector.  The  models  chosen  for  presentation  are 
those  MCDMs  that  predominate  the  literature  of  management, 
management  science  and  operations  research.  As  such,  they 
are  considered  the  most  useful  and  widely  recognized  for  R&D 
project  selection  and  other  resource  allocation  problems  of 
this  kind.  These  are  the  MCDMs  which  the  management  DM  in 
the  public  sector  is  most  likely  to  encounter  and  for  which 
some  knowledge  of  applications  will  be  increasingly 
important . 

We  begin  with  a  summary  of  some  of  the  graphical 
techniques  for  multi -criteria  decision  support  developed 
from  several  sources.  We  will  then  move  to  a  development  of 
the  Analytic  Hierarchy  Process  (AHP) .  Next,  the 


36 


Multiattribute  Utility  Theory  (MAUT)  will  be  detailed. 
Finally,  chapter  VIII  will  present  a  development  of  the  Goal 
Programming  (GP)  method  for  solving  problems  of  choice  with 
multiple  criteria. 

It  is  important  to  keep  in  mind  that  this  is  not  a 
technical  development  or  analytical  assessment.  As  a 
management  student  curious  about  the  power  and  limitations 
of  MCDMs,  my  goal  is  to  spark  the  curiosity  and  inform  rny 
peers  about  what  management  science  is  contributing  to  this 
specific  decision  environment.  As  in  an  explanation  of  the 
value  and  power  of  application  software,  the  point  is  what 
these  tools  can  do  for  us,  not  the  bits  and  bytes  of  how 
it's  done.  A  deeper  understanding  can  and  should  be  sought 
from  the  primary  sources  listed  as  the  references. 

Throughout  the  discussion  so  far,  several  terms  central 
to  decision  support  modeling  have  been  used,  the  meanings  of 
which  may  be  becoming  somewhat  blurred.  To  alleviate 
confusion,  the  following  definitions  are  offered  to  assist 
the  reader  in  keeping  each  term  in  focus: 

•  criteria:  the  measures  of  effectiveness  against  which 
alternatives  are  evaluated. 

•  attributes:  the  variable  outcomes  contributing  to  pro¬ 
gram  objectives. 

•  objectives/goals:  stated  and  implied  R&D  program 
mandates  and  priorities. 
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•  constraints:  the  physical  and  economic  limitations  to 
program  optimization. 

•  alternatives:  candidate  projects  for  selection. 
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V.  GRAPHICAL  APPROACHES  TO  MULTI -CRITERIA  DECISION  PROBLEMS 


We  begin  our  discussion  of  multicriteria  decision  models 
with  a  development  of  some  graphical  approaches  to  solving 
the  Army  R&D  selection  problem.  These  techniques  are 
adequately  covered  in  the  literature  of  management  science 
for  solving  choice  dilemmas  with  conflicting  attributes.  But 
more  significant  is  their  flexibility.  These  models  can 
incorporate  data  from  simple  cost/benefit  analysis  to 
sophisticated  risk  and  utility  functions  if  desired  by  the 
DM. 

First,  the  model  will  be  presented  through  a  discussion 
of  the  development  process  to  present  some  idea  of  how  the 
model  works  to  turn  input  data  into  output  recommendations. 
This  will  be  followed  by  a  discussion  of  the  technique's 
strengths  and  weaknesses  identified  through  research  or 
actual  applications  where  available.  The  chapter  will 
conclude  with  further  analysis  of  the  limitations,  pitfalls, 
and  strengths  of  this  technique.  Some  comparisons  between 
graphical  models  and  models  to  be  developed  in  later 
chapters  will  also  be  provided. 
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A.  GRAPHICAL  MODEL  STRUCTURE  AMD  FORMULATION 

Graphical  techniques  are  a  symbolic  depiction  of  the 
expected  outcome  for  each  alternative  with  respect  to  each 
attribute.  Subject  matter  experts  establish  the  measures  of 
effectiveness  criteria  upon  which  each  alternative  is  to  be 
judged  and  establish  some  effectiveness  rating  scale  for 
scoring  the  alternatives.  For  example,  some  objectives  for 
an  R&D  program  might  be  to: 

•  Maximize  utility  and  effectiveness  (coverage  such  as 
dual  use  technologies,  interoperability,  etc.) 

•  Maximize  research  personnel  utilization 

•  Minimize  development  time 

•  Minimize  resource  consumption  (funding,  overhead,  and 
support  costs,  for  example) 

•  Maximize  "critical  technologies"  state-of-the-art 

•  Minimize  development  risk  (such  as  outcome  suitability 
and  contractor  experience/ability  to  deliver) 

Next,  some  measures  of  effectiveness  (MOE)  for  each 

objective  would  have  to  be  established  and  agreed  to  by  the 

subject  experts/stakeholders.  Determining  the  program 

objectives  and  adequate  MOE  for  evaluating  each  alternative 

project  under  consideration  is  the  first  step  in  applying 

graphical  techniques.  [Ref.  28] 

Some  of  these  measures,  such  as  costs,  are  hard  criteria 

for  which  reasonable  data  figures  may  be  available.  Some  of 

the  others  are  soft  criteria  that  would  have  to  be  based  on 

expert  judgment,  experience  and  intuitions  of  those 
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involved.  These  soft  criteria  must  be  quantified  so  that 
measures  can  be  compared  on  an  equivalent  basis.  [Ref.  29] 
One  way  to  do  this  is  to  establish  "scoring  functions" 
or  graphs  which  depict  ratings  for  the  predicted  alternative 
attributes  (outcomes)  for  each  of  the  selected  criteria, 
such  as  those  shown  in  Figure  5.1.  Both  quantitative  and 
qualitative  results  can  thus  be  shown.  [Ref.  30] 


Figure  5.1  Example  Scoring  Function  Graphs 


One  graphic  technique  for  multiattribute  decision 

■ 

analysis  presented  by  Zeleney,  Canada  and  Sullivan,  and 
others,  [Ref.  29,  30  and  31],  is  called  Polar  Graphing.  This 
method  consolidates  the  results  from  scoring  function  graphs 


to  visually  demonstrate  relative  contributions  of  each 
alternative's  attributes.  Two  example  presentations  of  the 
model's  output  are  shown  in  Figure  5.2  [Ref.  29  and  30]. 


Figure  5.2  Typical  Polar  Graphs 


Polar  graphs  can  facilitate  decision  making  by  revealing 
those  projects  that  make  the  greatest  contribution  to  the 
RDEC  research  and  development  program  objectives.  As  with 
many  graphs,  however,  the  axis,  scale  values  can  be 
manipulated  to  represent  the  data  in  different  ways  for  a 
variety  of  interpretations . 

One  criticism  of  the  polar  graphing  technique  is  that  in 
structuring  this  method  of  multicriteria  decision  modeling, 
no  author  has  formulated  a  specific  way  to  weight  the 
criteria  (represented  by  the  "spokes*  in  Figure  5.2)  for 
evaluating  the  attributes  (depicted  as  the  "spider  web" 
linear  values  around  the  spokes)  of  the  various  alternatives 
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in  a  consistent  way.  Although  general  "utility  values"  can 
be  chosen  and  assigned  using  the  scoring  function  graphs, 
this  is  considered  highly  subjective.  While  the  model 
analyst  and  DM  may  develop  agreement  on  consistent  MOE, 
their  interpretation  and  the  resulting  axis,  scale  values 
might  not  hold  up  under  close  scrutiny.  [Ref.  32] 

This  issue  of  MOE  validity  for  the  selected  utilities 
and  their  representation  of  the  true  values  is  dependent,  at 
least  in  part,  upon  the  specific  program  objectives  and  how 
closely  the  utility  values  and  scores  represent  them.  If  the 
scale  values  are  accurately  represented,  the  graphs  may 
reveal  relative  desirability  between  alternatives.  Then,  as 
long  as  all  experts  and  stakeholders  agree  which  attributes 
should  dominate  the  selection  criteria,  a  valid  choice  can 
be  made.  [Ref.  26] 

Looking  at  the  example  in  Figure  5.2,  another 
shortcoming  is  evident.  Clearly,  with  more  than  five  or  six 
criteria  and  three  to  five  alternatives,  the  aid  this 
presentation  provides  is  rather  dubious.  It  would  do  little 
to  clarify  the  choice  considerations.  On  the  other  hand, 
consider  the  inadequacy  of  traditional  models,  such  as  cost- 
benefit  ratios,  that  simplify  the  problem  by  limiting  the 
analysis  of  alternatives  to  a  comparison  of  only  two 
criteria.  A  polar  graph  analysis  could  demonstrate  the  need 
for  greater  clarity  among  alternatives.  [Ref.  29] 
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Another  graphical  method  that  at  least  partially 
overcomes  this  visual  complexity  problem  is  suggested  in 
Keeney  and  Raiffa's  Decision  with  Multiple  Objectives  [Ref. 
27] .  Their  approach  depicts  the  attribute  scores  of  each 
alternative  in  a  "performance  profile, "  like  that  shown  in 
Figure  5.3.  In  this  technique,  the  vertical  lines  depicting 
the  attribute  scales  are  not  standardized  to  an  evaluation 
rating  or  scoring  function  as  with  polar  graphing. 


Figure  5.3  Example  Performance  Profile  Graphic 


However, the  authors  indicate  that  this  can  be  done  if  it 
clarifies  the  problem.  In  addition,  Keeney  and  Raiffa's  work 
covers  the  development  of  weighted  value  functions  for 
determining  preferences  between  criteria.  [Ref.  27] 

Graphic  techniques  used  to  support  multi-criteria 
decision  problems  are  useful  because  they  are  intuitive  by 
design.  The  information  presented  can  be  used  much  as  in  a 
traditional  trade-off  analysis  assessing  more  than  two 
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criteria.  The  support  staff  officer  can  show  a  lot  of  the 
information  quickly  and  clearly  without  a  lot  of  fanfare  or 
technical  jargon  to  explain  the  approach.  The  results  or 
output  supporting  the  decision  at  hand  can  be  largely  self 
explanatory.  This  is  particularly  true  for  comparing  self- 
evaluation  and  cross-evaluation  of  alternatives,  as  seen  in 
the  examples  [Ref.  26]. 

B.  ASSESSMENT  OF  THE  STRENGTHS,  LIMITATIONS,  AND  WEAKNESSES 

OF  THE  GRAPHICAL  TECHNIQUES 

The  first  comment  heard  regarding  the  value  of  the  polar 
graphs  or  the  performance  profiles  is  what  scale  values  are 
to  be  used  and  how  are  these  values  determined?  In  both 
cases,  Keeney  and  Raiffa's  development  of  value  functions 
and  utility  functions  can  be  used  [Ref.  27].  In  addition, 
more  familiar  decision  aids  can  also  be  incorporated,  such 
as  net  present  value,  internal  rate  of  return,  cost-benefit 
analysis,  and  many  of  the  other  financial  and  economic 
assessment  techniques.  Some  authors  suggest  adjusting  these 
traditional  value  equations  for  risk  and  time  factors  to 
further  enhance  their  usefulness  and  accuracy  [Ref.  33]. 

This  is  actually  an  important  strength  of  multicriteria 
graphical  techniques.  That  is,*  they  are  extremely  flexible 
as  mentioned  at  the  outset.  Many  of  the  other  MCDM  that  will 
be  developed  in  the  upcoming  chapters  do  not  incorporate  the 
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traditional  economic  analysis  formulas  which  may  in  part 
account  for  the  lack  of  use  of  MCDM  in  American  business  and 
industry . 

In  the  often  cited  surveys  by  Matthew  Liberatore  and 
George  Titus  regarding  MCDM  for  R&D  project  applications, 
none  of  the  40  respondents  from  29  "Fortune  500"  firms 
queried  in  1983  used  MCDM  in  their  R&D  project  selection 
process.  However,  net  present  value  and  cost-benefit 
analysis  were  reportedly  used  74  percent  and  62  percent  of 
the  time,  respectively.  Clearly,  for  a  model  to  be  seen  as 
effective,  it  must  first  be  regarded  as  important  and 
reliable  by  the  intended  user.  On  this  basis,  the  graphical 
approaches  must  be  given  high  marks  for  user  confidence. 

[Ref.  21] 

Another  shortcoming  cited  by  Vickers  and  Belton  in  their 
1988  study  on  graphical  techniques  (which  they  refer  to  as 
"visual  interactive  models")  is  the  time  and  difficulty  of 
formulating  these  models  and  their  accompanying  displays  for 
real  problems.  However,  the  authors  suggest  the  use  of 
several  commercially  available  software  packages  to  assist 
DMs  and  analysts  with  formulations.  They  recommend  Symphony 
and  Framework  for  the  novice  and  Why ,  Hocus,  Witness .  and 
Genetik  for  more  advanced  formulations  by  analysts.  The 
authors  also  present  a  complete  model  development  with 
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results  from  input  by  60  student  evaluators.  Their  work  is 
exceptionally  informative  and  helpful  for  the  beginner . [Ref . 
31] 

Dr.  Milan  Zeleny  of  the  Graduate  School  of  Business, 
Fordam  University  argues  effectively  that  polar  graphs  and 
other  graphic  techniques  are  more  than  mere  visual  displays 
because  they  serve  to  improve  decision  making.  He  points  out 
that  most  MCDM  ease  the  decision  process  by  separating  the 
problem  into  its  sub-parts  and  guiding  the  DM  to  state  his 
preferences  in  a  piecemeal  fashion. 

Zeleny  argues  that  this  only  reorganizes  the  problem 
rather  than  resolving  the  inefficiencies  resulting  from 
bounded  rationality.  He  explains  that  this  is  primarily 
because  such  methods  disrupt  the  holistic  view  of  the 
problem.  It  is  the  holistic  view  that  is  essential  to  good 
decision  making  and  the  essence  of  human  cognitive  skills. 
Graphical  techniques,  on  the  other  hand,  fit  all  the 
essential  pieces  of  conflicting  information  together  to 
diminish  the  role  of  the  analyst/facilitator  and  increase 
that  of  the  decision  maker.  [Ref.  29] 

In  conducting  this  research  it  appeared  that  another 
important  weakness  of  graphical  techniques  for  multicriteria 
decision  problems  is  the  lack  of  data  from  users  conducting 
actual  problem  analysis.  The  only  data  found  were  from 
hypothetical  modeling  of  simple  problems.  This  might  be  a 
result  of  the  fact  that  no  particular  author  or  group  claims 
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ownership  of  these  methods  to  champion  the  research  efforts 
necessary.  Additional  field  work  needs  to  be  done  to  prove 
the  theory  and  reassure  potential  users  of  the  benefits 
obtainable . 
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VI.  THE  ANALYTIC  HIERARCHY  PROCESS 


Chapter  VI  will  familiarize  the  reader  with  the  Analytic 
Hierarchy  Process  ( AHP)  for  aiding  the  DM  in  determining  the 
optimum  mix  of  R&D  projects  to  comprise  the  program  within  a 
limited  budget  environment.  Much  has  been  written  about  the 
strengths  and  weaknesses  of  AHP  and  its  applicability  to 
this  particular  problem.  This  chapter  will  present  an 
overview  and  description  of  how  the  model  is  formulated 
followed  by  some  assessments  of  AHP  from  users  of  the 
technique . 

A.  INTRODUCTION  TO  AHP 

The  Analytic  Hierarchy  Process  was  developed  and 
introduced  by  Professor  Thomas  L.  Saaty  of  the  Wharton 
School,  University  of  Pennsylvania  and  documented  in  his 
1980  book  of  the  same  name  [Ref.  34].  This  technique  is 
quite  possibly  the  most  important  for  quantifying 
qualitative  expert  opinion  and  judgment  for  decision 
support,  at  least  in  terms  of  its  ease  of  use,  its  parallel 
to  actual  decision  making  processes,  and  its  broad 
applicability.  AHP  enjoys  the  widest  acceptance  among 
managers  of  all  the  multi-criteria  decision  models  that  will 
be  presented  in  this  work. 
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This  technique  requires  the  DM  to  construct  a  hierarchy 


with  the  organization's  top  objective,  broadest  goal,  or 
mandate  at  the  top.  Subsequent  levels  of  the  hierarchy 
include  more  specific  objectives,  activities,  or  effects 
necessary  to  achieve  the  top  goal,  such  as  those  listed  up 
in  Table  4.1,  from  the  Army's  Technology  Base  Master  Plan 
[Ref.  6] .  The  lower  levels  are  the  attributes  of  specific 
alternative  project  choices  (courses  of  action)  that 
contribute  to  the  organization's  objectives.  The  final  level 
is  comprised  of  the  specific  projects  themselves.  An  example 
three  tiered  hierarchy  is  shown  below  in  Figure  6.1.  [Ref. 
30] 


Mandate : 


Obj ectives 


Enhance 

Warfighting 


Min 

Min 

Max 

cost 

Risk 

Benefit 

Max 

Max 

Capability 

Availability 

Figure  6.1  Sample  Decision  Hierarchy 

The  process  of  developing  this  decision  hierarchy  breaks 
the  decision  down  into  its  sub-elements  and  is  designed  to 
get  the  organization  to  concentrate  on  where  its  resources 
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need  to  be  focused  to  achieve  their  ultimate  goals.  There  is 
little  room  for  hidden  agendas,  nest  feathering,  or  the 
"underling  problem"  (wherein  a  subordinate  seeks  to  maximize 
personal  aspirations  regardless  of  their  impact  on  the 
organization's  goals)  when  priorities  are  examined  in  this 
process.  [Ref.  22,  p.43] 

The  hierarchy  development  process  can  also  disclose 
relations  and  overlap  of  goals  and  projects.  It  is  also 
hoped  that  the  process  will  help  to  examine  justifications 
in  terms  of  "virtues"  or  planning  criteria  of  future  returns 
on  R&D .  The  hierarchy  can  help  clarify  what  has  been 
omitted,  where  boundaries  have  been  drawn  sub-optimally 
narrow,  and  where  principles  and  programs  extraneous  to  the 
organization's  basic  mission  are  being  used  to  justify 
expenditures.  [Ref.  35] 

The  hierarchical  structuring  of  the  decision  problem  is 
to  be  developed  to  deal  with  three  issues  in  the  R&D 
investment  problem: 

•  How  do  we  plan  for  research  and  development  under  a 
multitude  of  uncertainties? 

•  How  can  we  design  and  implement  our  R&D  program  so 
that,  if  successful  in  its  separate  parts,  it  will 
aggregate  to  a  coherent  system  a  quarter  century  hence? 

•  How  can  the  organization,  given  a  quality  selection  of 
projects  that  emerge  from  our  analysis  of  the  above 
questions,  allocate  our  budget  cost  effectively  so  that 
benefits  of  investments  in  technologies  today  will 
accrue  on  schedule?  [Ref.  35] 
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Within  the  hierarchy  itself,  each  element  at  a  given 
level,  called  "virtues"  (or  attributes),  may  also  be 
assessed  as  obstacles.  As  the  hierarchy  is  developed,  the 
experts  can  evaluate  the  severity  of  the  challenges  posed  to 
operations  if  a  given  attribute  were  inadequately  resourced. 
The  intent  is  for  the  subject  experts  to  identify  areas 
where  future  funding  will  be  needed  to  stave  off  potential 
lags  in  capability.  This  is  to  orient  the  exercise  toward 
anticipating  growth  needs  rather  than  simply  extrapolating 
trends.  [Ref.  35] 

The  sub-objectives  of  the  hierarchy  can  also  be 
established  as  priorities  broken  out  by  the  organization's 
divisions.  The  DM  can  use  the  hierarchy  to  assess  the 
relative  merits  of  alternative  programs,  projects  and 
technologies.  What  emerges  is  a  hierarchy  of  benefits  that 
can  be  expected  to  accrue  to  each  program  area.  The 
potential  benefits  can  be  discounted  and  thus  used  to  plan 
current  investment  over  the  organization's  planning  horizon- 
one,  five,  or  more  years  out.  But  this  is  just  the  first 
step  in  the  AHP  analysis . [Ref .  35] 

Once  the  hierarchy  has  been  designed,  the  next  step  is 
to  elicit  expert  opinion  to  compare  the  attributes  on  each 
level  to  one  another  to  determine  and  establish  the  relative 
merit  of  each  attribute  in  terms  of  its  contribution  to 
achieving  the  next  higher  level  goal.  The  experts  assign  a 
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number  value  from  a  scale  like  the  one  shown  in  Table  6.1. 


These  scores  (called  preference  factors)  are  recorded  and 
organized  into  a  separate  matrix  for  each  level's  elements. 

TABLE  6.1 

SAMPLE  PREFERENCE  SCALE 


IF  attribute  1  is... 

preferred  to  attribute  2 

" Equal ly",  then  rank  is  1 


"Slightly"  =  3 
"Slightly  more"  =  5 
"Strongly"  =  7 
"  Always "  =  9 


The  scores  are  then  aggregated  and  standardized  using 
the  formal  theorems  of  matrix  algebra  to  derive  relative 
weights  for  the  projects  at  the  lowest  level  of  the 
hierarchy.  The  experts'  preferences  are  thus  translated  into 
a  project  selection  ranking  of  the  alternatives  with  respect 
to  the  attributes  by  revealing  which  projects  are  most 
adequate  for  achieving  the  overall  goal.  [Ref.  25] 

Although  the  matrix  algebra  calculations  can  get  fairly 

extensive  with  only  a  few  levels  of  attributes  and  experts 

« 

scoring,  software  is  commercially  available  to  conduct  the 
calculations.  The  use  of  a  computer  software  package  is 
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essential  for  quantifying  the  preference  value  judgments  of 
several  experts  from  different  departments  or  divisions. 
[Ref.  25]  1 

Each  expert  with  input  to  the  decision  process  must 
make  l/2n  x  (n-1)  paired  comparisons  (where  n  is  the  number 
of  elements  on  a  level  of  the  hierarchy) .  This  means  that  a 
hierarchy  with  ten  alternatives,  eight  attributes,  five 
objects,  and  a  primary  goal  would  require  83  paired 
comparisons  from  each  expert.  Because  of  this  need  for 
extensive  input  data  as  well  as  the  limits  of  human 
cognition  for  making  these  comparisons,  Professor  Saaty  and 
others  have  recommended  keeping  the  number  of  elements  in 
any  level  at  no  more  than  nine  and  the  number  of  levels  to 
between  three  and  five.  [Ref.  25  and  36] 

B.  ASSESSMENT  OF  THE  STRENGTHS,  LIMITATIONS,  AND  WEAKNESSES 

OF  THE  AHP  MODEL 

In  1984,  researchers  conducted  a  multi-year  study  of  an 
actual  application  of  the  AHP  method  to  the  R&D  selection 
process  of  a  pharmaceuticals  manufacturing  firm  in  England. 
The  application  was  conducted  to  assess  the  strengths  and 
weaknesses  of  AHP  against  those  of  other  multi at tribute 
decision  methods  for  an  actual  case.  The  study  was  not  an 

1  For  additional  information,  contact  Decision  Support, 
Inc.  about  "Expert  Choice"  at  4922  Ellsworth  Ave., 
Pittsburgh,  PA  15213  or  call  (412)  682-3844. 
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experiment  but  rather  an  application  of  AHP  in  the  public 
sector.  The  problem  required  ranking  ten  alternative 
projects  against  eight  criteria  attributes  selected  by  the 
firm's  DM  and  his  project  managers.  [Ref.  37] 

The  study  is  a  good  example  demonstrating  the 
formulation  process,  model  output,  and  user  reactions.  The 
decision  maker  and  his  staff  in  this  case  regarded  the 
process  of  using  the  preference  factor  ranking  scale 
confusing  and  the  extensive  paired  comparisons,  described 
earlier,  as  tedious.  Even  after  explaining  the  preference 
factor  to  the  users,  the  authors  reported  many  still  made 
errors  and  found  that  they  could  not  remember  how  they  had 
ranked  earlier  alternatives  to  attribute  comparisons  the 
further  they  got  into  the  procedure. 

The  researchers  were  able  to  work  the  participants 
through  these  formulation  problems  to  obtain  output  the 
participants  found  reliable.  In  spite  of  the  shortcomings 
they  had  cited,  all  of  the  participants  found  the  process  of 
structuring  the  decision  in  an  analytic  hierarchy  useful  for 
gaining  insight  and  understanding  of  the  important  factors 
in  research  planning.  In  fact,  the  authors  report  in  their 
conclusion  that  the  R&D  division  of  the  firm  has 
incorporated  AHP,  with  some  recommended  modifications,  into 
its  routine  R&D  selection  procedures. 

In  a  1987  follow  up  to  this  original  research  work, 

Geoff  Lockett  and  Mike  Stratford  found  the  R&D  division  at 


55 


ICI  Pharmaceuticals  continued  using  AHP  without  the 
technical  advisors  to  guide  them.  The  DM  was  utilizing  and 
acting  on  the  numerical  results  obtained  from  the  model.  The 
section  managers  and  project  manager  found  that  formulating 
the  decision  problem  with  AHP  was  helpful  and  the  output 
was  useful.  [Ref.  38] 

However,  Lockett  and  Stratford  also  stated  that  some  of 
the  respondents  (including  the  project  manager  himself)  were 
continuing  to  misuse  the  preference  factor  scale  and  changed 
their  input  choices  once  they  had  additional  clarification 
on  its  correct  use.  The  authors  indicated  that  this  was  a 
significant  drawback  to  AHP  since,  in  their  opinion,  "In 
practice,  judgmental  models  will  have  to  be  used  without 
technical  advisors  if  they  are  to  become  accepted  [Ref.  38, 
p.  399]."  In  addition,  after  conducting  a  comparison,  they 
recommended  incorporating  multiatribute  utility  theory 
(MAUT)  methods  for  choices  among  similar  attributes  to 
improve  results. 

Several  other  improvements  have  also  been  suggested  by 
researchers  working  with  the  AHP  technique  for  the  R&D 
selection  problem  [Ref.  39,  p.  305  and  p.  345]  In  addition. 
Professor  Saaty  has  responded  to  the  criticisms  regarding 
the  data  from  value  comparisons  using  his  original 
preference  scale  in  a  1987  article  in  Decision  Sciences.  He 
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suggests  ranking  alternatives  with  a  relative  proportion 
method  rather  than  his  original  scale  values  of  1-9.  [Ref. 
40] 

There  literally  seems  to  be  no  end  to  the  controversies 
surrounding  the  value  and  weaknesses  of  AHP  in  assisting 
decision  makers  to  evaluate  multiple  alternatives  with 
multiple  conflicting  criteria.  While  one  source  would  find 
the  hierarchic  structure  development  awkward,  others  said 
that  it  helped  their  understanding.  Some  decried  the  lengthy 
comparison  process  as  unnecessarily  tedious,  while  others 
argued  this  was  a  strength  of  AHP  through  its  development  of 
a  kind  of  "average  preference"  value.  In  any  case,  the 
overwhelming  consensus  in  the  literature  is  that  applying 
AHP  to  the  decision  process  improved  the  quality  of 
decision  making  for  the  DM.  But  the  method  is  not  without 
its  critics,  many  of  whom  advocate  alternative  MCDM  methods 
of  their  own  (see,  for  example.  Ref.  25,  26,  and  28.) 
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VII.  MULTIATTRIBUTE  UTILITY  THEORY 


Another  multicriteria  decision  model  which  has  proven 
useful  for  supporting  the  evaluation  of  projects  for  R&D 
selection  is  multiattribute  utility  theory  or  MAUT.  As  has 
been  the  pattern  thus  far,  the  chapter  is  organized  in  two 
parts.  The  first  will  introduce  the  basic  elements  of  the 
model  and  its  formulation  and  the  second  will  review  the 
important  literature  regarding  the  model's  strengths  and 
limitations.  The  goal  is  to  allow  the  reader  to^garner  some 
notion  of  the  value  of  this  technique  for  supporting 
decision  problems  with  multiple  objectives. 

A.  INTRODUCTION  TO  MULTIATTRIBUTE  UTILITY  THEORY 

This  important  multicriteria  decision  model  uses 
theories  from  mathematics  and  economics  regarding  utilities 
to  quantify  DM  preferences.  The  first  and  dominate  example 
of  this  technique  was  presented  by  Ralph  Keeney  of  Woodward- 
Clyde  consultants  and  Dr.  Howard  Raiffa  of  Harvard 
University  in  their  1976  book,  Decisions  with  Multiple 
Objectives:  Preferences  and  Value  Tradeoffs.  [Ref.  27] 

The  Keeney-Raif f a  MAUT  decision  model  uses  fairly 
advanced  mathematical  development  and  economics  manipulation 
to  evaluate  alternatives  against  multiple  criteria  and 
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arrive  at  the  optimal  course  of  action  to  achieve  the 
desired  outcome,  goal  or  objective.  Although  its  application 
to  the  R&D  project  selection  problem  is  not  as  well 
documented  as  that  of  AHP,  it  has  been  widely  applied  to 
many  types  of  selection  problems  with  multiple  qualitative 
and  quantitative  criteria.  It  must  therefore  be  included  in 
any  serious  discussion  of  this  class  of  MCDM. 

Like  AHP,  Multiattribute  Utility  Theory  (MAUT)  begins  by 
determining  the  overall  program  goal,  objectives, 
attributes,  criteria,  and  project  alternatives  and 
organizing  them  into  a  hierarchy.  The  notion  is  to  divide 
the  multicriteria  problem  into  its  basic  component  parts, 
determine  the  DM's  preferences  on  each  component,  and  derive 
the  alternative  that  best  fulfills  those  preferences.  The 
process  can  become  fairly  complicated.  The  following  is  a 
narrative  explanation  of  the  steps  in  the  MAUT  procedure. 

Once  objectives,  attributes,  and  alternatives  are 
determined,  subjective  measures  of  effectiveness  are  derived 
and  assigned  to  each  attribute.  Objective  data  are  used  for 
this  whenever  available  and  applicable.  The  DM's  quantified 
values  and/or  expected  values  are  thus  established  for  each 
attribute.  Value  functions  are  then  derived  by  summing  the 
attribute  values  that  represent  the  DM's  preference 
structure  by  assigning  a  weighting  constant,  an  exponent,  or 
both  to  each  attribute  element  of  the  function  for  each 
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alternative  [Ref.  41].  Several  example  value  functions 
adapted  from  those  presented  by  Keeney  and  Raiffa  are  shown 
in  Figure  7.1  [Ref.  27,  p.81]. 


v(X)  =c1x1+c2x2+c2x2 
V(  Y)  -yly$y% 

v(Z )  =c1z1+c2z2+c3  {z1-b1)2  (z2-jb2)4 


Figure  7.1  Sample  Value 
Functions  for  Hypothetical 
Alternatives  X,  Y,  and  Z 


Next,  value  tradeoffs  are  conducted  systematically  in 
which  the  DM  must  decide  how  much  of  one  objective  he  is 
willing  to  give  up  to  improve  his  chances  of  achieving  one 
of  the  other  objectives.  Outcome  probabilities  are  used  to 
determine  expected  values  where  uncertainties  are  involved. 
The  tradeoff  analyses  are  conducted  between  the  alternative 
value  functions  using  marginal  analysis  as  in  classical 
economics.  The  basis  for  this  approach  is  that  the  utility 
of  any  course  of  action  under  risk  equals  the  expected  value 
of  its  payoff. 

Multiatribute  utility  theory  relies  heavily  on  economic 
principles  of  Pareto  optimal  analysis.  A  fundamental 
assumption  of  this  analysis  is  that  for  each  act  there  is  an 
outcome,  and  the  set  of  all  the  available  alternative 
outcomes  which  are  most  efficient  comprises  an  efficiency 
frontier  for  a  particular  decision  maker.  If  we,  as  that  DM, 
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knew  which  acts  (alternative  programs)  led  to  the  most 
efficient  outcomes,  and  we  were  rational,  we  would  choose  to 
do  those  acts  to  obtain  the  maximum  benefits.  That  is,  we 
would  choose  the  Pareto  optimal  set  of  alternatives.  With 
MAUT,  the  analysts  use  probability  and  DM  preferences  to 
derive  the  DM's  Pareto  optimal  outcome  set.  [Ref.  27] 

The  value  functions  and  the  outcome  probabilities  are 
combined  to  obtain  a  utility  function  for  each  alternative. 
The  alternatives  with  the  greatest  utility  for  the  DM  are 
those  that  should  comprise  his  R&D  program.  Numeric  values 
can  be  determined  for  the  attributes  comprising  the  utility 
functions  so  that  a  utility  value  can  be  calculated.  Thus 
the  alternative  projects  can  be  rank  ordered  from  greatest 
to  least  utility  for  achieving  the  organization's  goals. 

[Ref.  42] 

Obviously,  the  mathematical  expressions  get  rather 
lengthy  with  only  a  few  attributes  for  each  alternative.  Not 
only  that,  but  the  utility  functions  involve  the  integration 
of  the  area  under  the  theoretical  Pareto  optimal  curve  and 
the  curves  can  be  monotonically  decreasing  or  nonmonotonic 
increasing  in  three  space  with  additive  or  multiplicative 
risk  values!  This  is  not  the  kind  of  analysis  the  typical 
management  decision  maker  is  comfortable  with.  The  tradeoff 
analysis  process  for  just  three  alternatives  takes  the 
authors  nine  pages  to  explain.  [Ref.  27] 
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Utility  theory  says  that  the  DM  must  assign  a  utility  to 
a  course  of  action  (project  alternative)  based  on  his 
perception  of  the  risk  of  the  probable  outcome  for  each 
alternative.  For  example,  if  the  most  desired  outcome  were 
only  half  as  likely  to  occur  at  the  conclusion  of  project  A, 
but  a  less  desired  outcome  were  sure  to  occur  with 
alternative  B,  the  DM  should  assign  a  higher  utility  to 
project  B,  and  fund  B  accordingly.  The  point  is,  the  DM 
should  act  to  maximize  his  expected  utility.  The  DM's 
perception  of  risk  is  factored  in  as  well.  If  the  DM  is  risk 
prone  or  risk  averse,  he  will  naturally  adjust  his  utility 
for  each  alternative.  [Ref.  22] 

With  MAUT  the  assumption  is  that  DM  preferences  are 
consistent  and  independent  of  each  other.  The  idea  of 
decomposing  a  problem  implies  that  there  will  be  less  error 
from  random  judgments  when  the  DM  can  focus  on  single 
components  of  a  decision  problem.  The  central  assumptions 
are  that  the  DM  will  make  rational  choices  that  optimize  his 
utility  and  that  a  decision  is  rational  if  it  maximizes  the 
DM's  expected  utility. 

One  other  point  unique  to  MAUT  is  that  the  technique 
allows  for  only  one  DM  instead  of  inputs  from  multiple 
experts.  Utilities  of  multiple  stakeholders  are  not 
aggregated  and  factored  in  mathematically,  but  their 
opinions  should  be  weighed  into  the  DM's  utility  function 
for  each  alternative.  The  DM  could  call  for  subject  experts 


62 


to  provide  input  to  set  attribute  values  and  assist  with 
utility  tradeoffs  preferences.  It  would  then  be  up  to  the  DM 
how  he  would  use  their  input  to  weigh  on  his  own  judgments. 
[Ref.  41] 

B.  ASSESSMENT  OF  THE  STRENGTHS,  LIMITATIONS,  AND  WEAKNESSES 

OF  THE  MAUT  MODEL 

First  of  all,  do  not  feel  intimidated  if,  having  read 
the  above  explanation,  you're  not  sure  what  you  read.  This 
seems  to  be  a  frequent  response  to  utility  theory  in  general 
and  MAUT  in  particular.  In  many  instances  this  was  the 
typical  user  response.  In  comparisons  with  other  techniques 
such  as  AHP  and  goal  programming,  the  outcomes  for  optimal 
alternative  ranking  were  strikingly  similar.  This  often 
reassured  users  of  the  method's  value.  However,  the  concern 
remained  that  this  model  may  be  difficult  to  use  without  the 
assistance  of  a  competent  analyst  to  work  the  DM  through  the 
process  'see,  for  example.  Ref.  37  and  41). 

This  lack  of  understanding  of  how  the  MAUT  quantifies 
qualitative  preference  "data"  and  probable  outcome  functions 
to  obtain  its  rank  order  solution  is  the  complaint  most 
common  in  application  analyses  uncovered  in  this  research. 
Many  of  the  researchers  have  questioned  the  value  of  MAUT  on 
the  basis  of  this  concern,  pointing  out  that  the 
formulation  itself-  the  way  value  and  preference  questions 
are  themselves  presented-  may  too  strongly  influence  the 
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optimal  outcome  results.  More  study  should  be  done  in  this 
regard,  but  some  research  has  already  shown  that  this  point 
does  not  negate  the  value  of  MAUT  to  decision  makers  in 
actual  practice.  [Ref.  43] 

One  of  the  other  common  complaints  about  MAUT  is  the 
lack  of  a  means  for  sensitivity  analysis.  The  value 
functions  and  probability  analysis  leading  to  the  utility 
functions  that  comprise  the  model  and  determine  the  output 
are  not  easily  manipulated  to  answer  DM's  various  "what  if" 
questions.  In  addition,  if  the  value  functions  developed  do 
not  accurately  reflect  the  actual  Pareto  optimal  efficiency 
frontier,  the  output  results  will  reflect  the  inaccuracy  but 
without  the  knowledge  of  the  DM.  As  one  author  has  remarked, 
"For  all  its  outward  appearance  of  mathematical  precision, 
MAUT  is  highly  subjective,  is  not  intuitive,  and  presents 
many  difficulties  in  practice."  [Ref.  30,  pp. 256-257] 
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VIII.  THE  GOAL  PROGRAMMING  MODEL 


With  this  chapter  the  introduction  to  the  primary, 
important  multicriteria  decision  models  for  R&D  project 
selection  is  completed.  The  goal  programming  ( GP }  method 
will  be  explained  here  through  a  description  of  the 
formulation  procedures.  This  will  be  followed,  as  in  the 
preceding  three  chapters  with  a  summary  of  the  strengths  and 
limitations  of  GP  for  solving  multicriteria  problems  such  as 
those  facing  the  Army's  R&D  community. 

A.  INTRODUCTION  TO  GOAL  PROGRAMMING 

The  Goal  Programming  (GP)  technique  for  supporting 
multiattribute  decision  making  is  a  modification  of  linear 
programming,  using  mathematics  to  spread  limited  resources 
(time,  manpower,  funding,  etc.)  over  multiple  objectives. 
Abraham  Charnes  and  William  Cooper  are  frequently  credited 
with  first  introducing  this  method  in  1961.  [Ref.  30] 

Because  of  its  longevity,  many  complete  formulations  of 
the  GP  model  for  solving  the  R&D  selection  problem  in  the 
public  sector  are  available  for  further  study  (see,  for 
example,  References  36,  44,  and  45).  Nevertheless,  in  their 
1983  survey  of  29  major  "Fortune  500"  industrial  firms, 
Liberatore  and  Titus  found  only  four  of  the  40  respondents 
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indicated  that  they  were  even  aware  of  GP  and  none  indicated 
that  they  used  the  technique  to  support  their  R&D  project 
selection  decisions.  [Ref.  21] 

B.  GOAL  PROGRAMMING  MODEL  DESCRIPTION 

With  linear  programming  the  DM  seeks  to  optimize  a 
single  objective  given  the  constraints  imposed  by  the 
environment  in  which  the  decision  is  made.  It  is  very  widely 
used,  for  example,  in  scheduling  machine  operating  hours  for 
various  jobs.  This  is  a  well  known  example,  where  the 
constraints  might  be  time  and  materials  with  the  objective 
of  maximizing  production  output  to  meet  demand. 

With  GP  on  the  other  hand,  the  object  is  to  satisfy  a 
number  of  competing  objectives  as  completely  as  possible 
with  the  given  resource  constraints.  As  in  our  production 
example,  we  might  set  maximum  production  output  as  a  goal 
along  with  say,  minimizing  operator  overtime  (accounting  for 
worker  safety  and  fatigue  as  well  as  expenses) ,  minimize 
machine  set-up  times  (caused  by  switching  from  one  type  of 
job  to  another) ,  and  minimize  finished  goods  inventory 
stack-up  at  the  next  work  station.  [Ref.  22] 

Each  of  these  goals  competes  for  the  critical  resources 
of  time  and  materials  which  must  be  conserved  while  filling 
each  of  the  competing  demands  as  completely  as  possible. 
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This  is  a  classic  GP  problem.  Because  of  this  capacity  for 
handling  multiple  objectives,  GP  has  been  established  as  one 
of  the  dominate,  if  not  the  predominate  MCDM. 

C.  THE  GOAL  PROGRAMMING  MODEL  FORMULATION  PROCESS 

The  goal  programming  technique  is  powerful  because  once 
the  problem  is  formulated  into  its  linear  objectives  and 
constraints,  a  sound  feasible  solution  can  be  obtained  for 
the  DM  which  is  intuitive  and  logically  supportable.  The 
model  is  also  appealing  because  it  is  flexible. 

Goals  may  be  stated  as  absolute  priorities,  as 
objectives  to  be  sought  in  some  ranking  by  priority,  or 
interval  goal  constraints  can  be  specified  with  over  or 
under  achievement  boundaries.  Although  the  DM  may  state 
preferences  between  competing  objectives  as  with  AHP  or 
MAUT,  there  is  no  requirement  to  do  so.  [Ref.  22] 

Weighting  constants  are  not  used  in  GP.  Instead  the 
objective  functions  contain  plus  and/or  minus  deviation 
variables,  which  represent  surplus  or  slack  resources  that 
can  be  applied  to  competing  objectives  in  the  formulation. 

A  negative  deviation  variable  represents  a  floor  limit  set 
by  the  DM,  while  a  positive  deviation  variable  is  a  ceiling. 
Both  plus  and  minus  deviation  variables  (the  dj+"  factors  in 
equation  7-3,  of  Figure  8.1)  in  one  objective  function 
designate  an  interval  for  required  fulfillment.  If  the 
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objective  is  absolute,  no  deviation  variables  are  included 
in  that  objective  function.  [Ref.  44] 

If  absolute  priorities  are  established  for  more  than  one 
objective  in  the  formulation,  the  goals  must  be  ranked  in 
the  DM's  order  of  importance.  Preferences  can  be  expressed 
in  many  ways,  but  AHP  is  often  recommended  if  a  number  of 
tradeoff  comparisons  must  be  determined.  [Ref.  22] 

Once  linear  objective  functions  are  constructed  from  the 
DM  preferences,  the  GP  technique  uses  a  preemptive  goal 
algorithm  to  maximize  each  objective.  This  means  that  lower 
importance  goals  are  only  considered  after  all  higher 
objectives  are  fulfilled.  Priority  coefficients  (the  P  k 
factor  in  equation  7-2  in  Figure  8.1)  can  also  be 
established  to  indicate  priorities  where  slack  or  surplus 
resources  should  be  assigned  by  DM  preference. 

Find.  .  ,X=  {x1,x2,xi .  .  .  ,xn)  (7  -1) 

Maximize.  .  .  Q=JT  Pk-f  (dj ,  dj)  (7-2) 
s.t - 52  aijxj+dJ~di=ci  (7  ~3 ) 

Figure  8.1  Example  Goal  Programming 

Formulation 

Figure  8.1  contains  a  generic  GP  formulation  that 
includes  each  of  the  elements  discussed  so  far.  In  this 
example  formulation,  the  Xj  factors  are  the  project 
alternatives,  the  ai;j  is  a  coefficient  value  associated  with 
a  particular  project  (a  quantity  of  resource  j  for 
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alternative  i,  for  example),  and  c  .j  is  the  goal  or 
objective  value  for  constraint  j.  [Ref.  30,  pp. 289-290] 

As  this  explanation  may  indicate,  formulation  of  a 
multiple  objective  decision  problem  with  competing 
priorities,  and  constraints  into  a  mathematical  model  is  no 

*  simple  task.  Goal  programming  is  a  recipe  for  transforming  a 
problem,  such  as  R&D  project  selection,  into  a  model  from 
which  a  solution  can  be  derived. 

To  summarize,  there  are  three  basic  steps  in  the 
formulation  process: 

•  Determining  the  aspiration  levels  for  goal  constraints 

•  Determining  the  slack  or  surplus  allowed  for  each 
objective  and  the  priority  (if  any)  by  which  each  is  to 
be  allotted  to  competing  objectives,  and 

•  Development  of  the  objective  functions. 

D.  ASSESSMENT  OF  THE  STRENGTHS,  LIMITATIONS,  AND  WEAKNESSES 
GOAL  PROGRAMMING  MODEL 

Although  formulation  of  the  specific  GP  model  itself  can 
be  formidable  when  each  step  is  straight  forward,  there  are 
two  assumptions  that  must  be  dealt  with  even  before 
formulation  can  begin.  First,  each  objective  function 
derived  will  be  linear  and  second,  all  goal  constraints  must 
be  quantifiable  for  mathematical  formulation. 

•  Both  of  these  assumptions  must  be  understood  and  agreed 
to  by  *-v>e  DM  b^ore  the  objectives  can  be  quantified  and 
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formulated  into  a  value  function.  Take  for  example  an 
objective,  "to  minimize  program  risk." 

Development  risk  should  decrease  over  time,  as 
complications  are  worked  out  of  the  design  approach. 

Schedule  risk,  on  the  other  hand,  may  increase  or  decrease 
several  times  as  the  project  continues,  but  it  will  most 
certainly  not  remain  linear.  In  either  case  assumed 
linearity  of  the  risk  factor  may  or  may  not  be  significant 
enough  to  invalidate  the  model's  output.  After  all,  it  is 
only  a  model.  But  generalizations  of  this  kind  can  adversely 
effect  DM  confidence  in  the  output  reliability,  thereby 
virtually  invalidating  the  entire  modelling  effort. 

[Ref.  45] 

As  for  quantification,  quantifying  constraints  like 
time,  funding  levels,  personnel  and  materials  is  fairly 
routine  in  a  GP  formulation.  If,  however  the  DM's  objectives 
are  to  "maximize  benefit,  capabilities,  or  value  of  the 
research  efforts",  GP  alone  will  not  suffice.  [Ref.  46] 

One  way  to  quantify  the  intangibles  is  to  employ  other 
techniques,  such  as  AHP,  to  quantify  qualitative  factors  and 
use  the  output  as  input  to  GP  so  that  the  problem  can  be 
formulated.  An  example  of  combining  the  two  procedures  was 
conducted  for  the  U.S.  Army  Strategic  Defense  Command  in 
1987  to  determine  R&D  project  selection  given  the  program's 
multiple  goals  and  constraints.  [Ref.  36] 
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One  important  strength  of  this  model  is  its  ability  to 
handle  a  vast  number  of  constraints  and  objectives  to 
evaluate  numerous  alternatives.  In  Dr.  James  Ignizio's  work 
for  the  Army  on  R&D  project  selection,  his  goal  programming 
code  handled  problems  with  over  100  variables,  over  20 
constraints,  and  from  three  to  five  objectives  [Ref. 

44,  p.  271].  Captain  Anderson  obtained  reliable  output  for 
the  Army  Strategic  Defense  Command  using  his  formulation  of 
five  objectives  and  35  constraints  to  evaluate  35 
alternative  projects  [Ref.  36,  pp. 49-59].  However,  there  is 
a  limit. 

In  their  GP  work,  Choo,  Wedley  and  Lam  reported  that 
their  model  "exploded"  (a  computer  term  for  an  endless  loop 
which  yields  no  feasible  solution)  with  any  evaluation  of 
over  53  alternatives.  The  authors  also  pointed  out  however, 
that  this  figure  was  a  matter  of  the  software  program  used, 
the  number  of  priority  coefficients  used  in  the  objectives, 
and  the  extensive  outcome  probabilities  used  for  each 
alternative.  [Ref.  47] 

Finally,  some  researchers  have  argued  effectively  that 
the  assessment  of  the  DM  preferences,  priorities,  and 
deviation  variables  in  the  GP  formulation  process  is 
actually  a  search  for  the  Pareto  optimal  alternative,  as 
described  in  chapter  VII.  They  point  out  that  ignoring  this 
fact  can  lead  to  suboptimal  results.  Halme  and  Korhonen,  for 
example,  develop  the  theory  (with  proofs)  for  incorporating 
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utility  theory  into  the  GP  analysis  in  their  1988  work  on 
multiple  objective  optimization  problems.  [Ref.  48] 

These  references  to  research  on  the  integration  of 
several  models  is  added  to  alert  the  reader.  Overlap  between 
the  dominate  MCDMs  may  be  encountered  in  actual  decision 
applications.  This  research  effort  indicated  that  successful 
applications  blended  the  strengths  of  at  least  two  models. 
Although  this  strategy  may  increase  formulation  time,  the 
results  have  been  encouraging. 

Goal  programming  is  considered  the  first  true  MCDM.  It 
has  been  evaluated  and  assessed  widely,  particularly 
overseas,  over  the  past  three  decades.  Almost  without 
exception,  the  response  to  its  results  for  improving  the 
quality  of  decisions  is  overwhelmingly  positive.  However, 
the  results  of  Titus  and  Liberatore's  survey  of  top 
management  from  29  "Fortune  500"  companies  in  1983  revealed 
that  not  only  did  American  business  not  use  this  technique 
for  R&D  management,  over  85  percent  indicated  that  they  were 
unaware  of  it.  This  indicates  a  clear  disconnect  between  the 
management  science  model  developers  and  the  managers  who  are 
the  users  of  these  techniques.  [Ref.  21] 
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IX.  CONCLUSION  AND  SUMMARY 


There  are  many  quantitative  techniques  available  to 
aid  the  decisionmaker  (sic)  in  arriving  at  a  choice  that 
has  a  high  probability  of  meeting  his  objective....  The 
military  decisionmaker  must  consider  the  ideas  and 
approaches  that  decision  technology  has  to  offer.  [Ref. 

49]  U.S.  Army  War  College,  Army  Command  and  Management . 
August  1991 

A.  CONCLUSION 

The  decision  models  presented  in  the  last  four  chapters 
represent  the  state-of-the-art  in  American  management 
science  efforts  to  assist  decision  makers  solve 
multicriteria  decision  problems.  None  of  them  will  replace  a 
decision  maker  or  diminish  the  responsibility  for  the 
decision  made.  Each  of  them  is  after  all,  a  representation, 
a  model  of  a  non-trivial  decision  process.  They  help  to 
expand  the  bounds  of  rationality  for  those  involved  in  the 
decision  by  prompting  an  analysis  of  as  much  of  the 
available  information  as  the  decision  maker  desires. 

Each  of  the  four  modeling  techniques  represents  a 
slightly  different  view  of  how  complex  decisions  are 
structured  to  get  a  handle  on  and  resolve  the  critical 
issues.  For  the  Army  Research,  Development,  and  Engineering 
Centers,  the  problem  is  how  to  allocate  limited  resources 
over  an  ever  widening  range  of  technological  opportunities 
to  achieve  each  of  their  goals  to  the  greatest  extent 
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possible.  This  is  a  common  problem  that  many  of  the  authors 
and  researchers  identified  herein  have  been  working  on  for 
business,  industry,  and  government  for  several  decades. 

These  models  are  not  static,  but  are  continually  being 
reshaped  to  meet  the  needs  of  the  user  community.  The  users 
are  international  in  scope,  from  Europe  to  the  Pacific  Rim 
and  Australia.  In  fact,  the  models  introduced  in  this  thesis 
seem  to  be  enjoying  their  greatest  acceptance  and 
application  in  Europe  and  Japan,  according  to  the  work  at 
the  International  Conference  on  Multiple  Criteria  Decision 
Making  [Ref .  39]. 

Decision  theory  in  general  and  MCDM  in  particular  will 
grow  increasingly  important  to  decision  makers  as  the  models 
become  more  sophisticated  and  user  oriented.  In  1983,  The 
Harvard  Business  Review  quoted  Robert  Newman  of  General 
Electric : 

Within  10  years,  decision  theory,  should  occupy  the  same 
role  for  the  manager  as  calculus  does  for  the  engineer 
today.  The  engineer  of  Roman  times  was  unaware  of 
calculus,  but  he  could  make  perfectly  good  bridges. 
However,  he  could  not  compete  today,  even  in  bridge 
building,  let  alone  in  astro-engineering.  Management 
today  is  at  the  stage  of  Roman  engineering.  Needless  to 
say,  managers  will  still  use  specialists,  just  as 
engineers  use  heat  transfer  experts.  [Ref.  50,  p.149] 

This  was  not  exceedingly  prophetic,  but  rather  a  statement 

of  the  obvious.  Decision  models  may  not  be  as  important  as 

calculus  yet,  but  their  potential  benefits  for  improving 

decision  quality  are  enormous. 
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B.  SUMMARY 


For  the  Array  research  and  development  project  selection 
problem,  MCDM  hold  a  great  deal  of  promise.  With  increased 
pressure  from  Congress  for  the  Department  of  Defense  to 
assure  America's  technological  defense  superiority  in  the 
face  of  dramatically  reduced  defense  spending,  there  will  be 
increased  scrutiny  and  accountability  for  the  Army's  R&D 
resource  commitment  decisions.  Multicriteria  decision  models 
offer  a  means  to  develop  logically  supportable  solutions  to 
this  issue. 

The  Army  decision  maker  will  rarely  know  with  absolute 
certainty  which  R&D  alternatives  hold  the  greatest  promise 
for  achieving  the  Army  investment  strategy  objectives.  The 
final  choice  will  likely  be  something  of  a  compromise,  a 
tradeoff  among  judgments  based  on  quantitative  and 
qualitative  factors  and  valued  expert  opinion  on  potential 
benefits,  risks,  costs,  and  payoffs. 

These  intuitions  from  competing  stakeholders  must  be 
assessed  on  a  level  basis,  as  free  as  possible  from  human 
biases,  and  in  keeping  with  organizational  values.  Multi¬ 
criteria  decision  models  can  incorporate  these  elements, 
organize  the  decision  criteria,  and  quantify  subjective 
opinion  regarding  alternatives'  relative  merits  so  that  the 
DM  can  make  a  rational  choice  on  the  basis  of  available 
data . 
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Furthermore,  the  limits  of  human  cognitive  skills 
introduce  additional  inefficiencies  of  their  own. 
Multicriteria  decision  models  can  bring  structure  to  these 
non-trivial  decisions  and  systematically  aid  the  decision 
maker  to  overcome  the  random  errors  common  to  multicriteria 
problems  of  choice.  Clearly  if  a  decision  problem  can  be 
effectively  modeled,  then  the  quality  of  the  decision  can  be 
improved . 

In  sum,  the  quality  and  efficiency  of  the  Army  R&D 
resource  commitment  must  increase.  The  decisions  to  commit 
resources  to  achieve  organizational  goals  and  execute 
mandates  are  perhaps  the  single  most  important  function  of 
any  leader.  Nevertheless,  precious  little  time  can  be 
dedicated  to  a  comprehensive  analysis  of  all  of  the  factors 
that  support  these  decisions.  Multicriteria  Decision  Models 
clearly  offer  an  attractive  means  for  resolving  this 
dilemma . 
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